Physical AI Lab // Episode 3.4 // One-pager
Long tasks without falling apart
Memory, planning and multi-policy routing: split a long job into manageable pieces
Three key points
- Long tasks are hard because errors accumulate: each step has a high chance of being right, but those chances multiply over dozens of steps, so the chance of getting every step right keeps falling, and drift keeps growing, so a single policy struggles alone.
- Layers decouple thinking and doing: the slow upper layer (GR00T's System Two does task breakdown) thinks about the big picture; the fast lower layer emits actions in real time. AWS measured π0 on an H100 with 1 ODE step: a chunk of 50 steps covers about 1 second, is generated in under 200 ms, so it can replan at about 5 Hz (O). These are π0 figures under that setup, not a property of every fast layer.
- The orchestrator handles multi-policy routing and memory: SayCan uses skill value functions, Hi Robot lets a VLM think directly (R); π0.5 first predicts a semantic subtask; evaluate per-step and whole-chain success separately.
Evidence: O official · R research · M media
| R | Two systems | GR00T N1 uses a dual-system architecture: a vision-language module (System Two) understands the scene, and a diffusion module (System One) generates actions in real time |
| O | System Two | of GR00T N1.7 (Cosmos-Reason2-2B backbone) is exactly where task breakdown and multi-step reasoning happen |
| O | < 200 ms | On an H100 with 1 ODE step, π0 generates a whole 50-step chunk in under 200 ms (AWS measurement) |
| O | ≈5 Hz | π0 computes a chunk in under 200 ms, so it can replan at about 5 Hz; each chunk covers about 1 second of motion (H100) |
| R | 3 kinds of robot | Hi Robot uses a layered VLM: the upper layer works out the next step, the lower layer carries it out; tested on single-arm, dual-arm and mobile dual-arm robots |
| R | SayCan | The paper reports: the language model supplies step-by-step knowledge for long tasks, and the skills' value functions judge whether a step can be done in the current environment |
| R | π0.5 | The paper reports: it predicts a semantic subtask, then actions; trained on mixed data, it tidies kitchens and bedrooms in entirely new homes (authors' own results) |
| O | 10 kinds | LIBERO-10 is a simulated tabletop manipulation benchmark with 10 task categories; this episode's experiment is a two-dimensional CPU stand-in, and LIBERO was not run here |
Further reading
- GR00T N1: a dual-system VLA (arXiv 2503.14734)
https://arxiv.org/abs/2503.14734 - Hi Robot: open-ended instruction following with a hierarchical VLA (arXiv 2502.19417)
https://arxiv.org/abs/2502.19417 - π0.5: a VLA with open-world generalisation (arXiv 2504.16054)
https://arxiv.org/abs/2504.16054 - SayCan: grounding language models in robotic affordances (arXiv 2204.01691)
https://arxiv.org/abs/2204.01691