Physical AI Lab // Episode 3.6 // Quiz

Reading the frontier: checking the numbers

Use the daily radar to recompute a paper's key numbers

01A report says fine-tuning gave "about a 96% improvement". Checking it, you find some of the evaluation episodes also appear in the training set. What is the most reasonable judgement?

02The same "90% success rate" appears in a vendor press release, an arXiv preprint and a tech-media article. Using the radar's evidence levels, which reading is right?

03You wrote a training script assuming "this dataset has 500 episodes", and halfway through it fails with an index-out-of-range error. What should you do first?

04In a report, two methods' rankings differ only slightly, and the numbers are averaged over just the first 5 held-out episodes. How should you put it in your check notes?

05When checking "action error fell by about ninety percent", which record is the most acceptable?