Physical AI Lab // Episode 1.7 // Quiz
Reinforcement learning Getting better by trial and error
No demonstrations, only rewards. Let it find the way itself
01
Someone says: “Reinforcement learning means showing the robot lots of demonstrations and then scoring every action.” What is wrong with this?
The scoring part; RL uses no rewards at all
Nothing; RL is imitation plus scoring
The scoring part; you can only score once, at the very end
The demo part; RL gives no demos, only rewards and trials
02
In one episode the robot gets a few points as its first steps bring it closer to the cup, but at the end it knocks over the bottle and loses a lot. What should it care about when learning?
The few points from the very first step
The total reward over the whole run
The score of the step closest to the cup
Only the big penalty at the very end
03
A team plans to have one real robot learn to walk from scratch by trial and error. By the episode's account, what advice would they most likely get?
Record demonstrations first, or it will never learn
If the reward is right, a few real tries are enough
Train many copies in parallel in simulation first
Fit tougher motors so it can fall as often as it likes
04
During training the return curve rises halfway and then stalls; in replays the policy always uses the same fairly safe move. What should you check first?
Exploration: is it stuck on the safest move?
There are too many reward terms; delete them all
Too many parallel copies; cut them down to one
A flat curve means it has learned the best it can
05
The reward only checks whether the cup moved. In a replay the robot swats the cup off the table, yet it scores high. What does this show?
The policy learned well, it's just a bit rough
The reward is flawed and it is gaming it
The gap between simulation and reality is too big
It still hasn't had nearly enough tries
Check