Physical AI Lab // Episode 3.3 // Quiz

RL post-training: the self-improvement loop

From imitation to trial and error, better with practice

01A coffee-pouring policy is trained on many successful demonstrations. In a test its hand shakes, coffee spills, and then it freezes. What is the best explanation?

02Compared with simply collecting more demos and imitating them, what does RECAP (π*0.6) add?

03Someone says: "HIL-SERL reached near-perfect success in 1 to 2.5 hours, so real-robot RL takes only a few hours for any robot task." Which point in the video most directly rebuts this?

04In simulation you train an arm with RL to push a block to a target. Return keeps rising, but the replay shows it just shoves the block against the table edge and never reaches the target. What is the most likely cause?

05This episode's experiment ran 3 training seeds. The 95% confidence intervals of the paired differences across seeds: return 13.48 to 19.74, success rate −9.0% to 57.9%. By the episode's criterion, what is the conclusion?