Physical AI Lab // Episode 3.1 // One-pager
Inside VLAs: three models
Vision encoder, language backbone, action head
Three key points
- Same skeleton: all three VLAs look at images, understand language, then output actions directly, with three blocks inside: seeing the world, understanding language, generating actions.
- Three action-head designs: OpenVLA splits actions into 256 bins as tokens (R); π-zero uses flow matching with an action expert, and with action chunking runs control at up to 50 hertz (R); GR00T N1's fast system is a diffusion transformer, also trained with flow matching, 120 hertz (R).
- Different backbones: OpenVLA = Llama 2 + DINOv2/SigLIP, 7 billion parameters (R); π-zero = PaliGemma, about 3.3 billion (R); GR00T N1 = Eagle-2 (SmolLM2 + SigLIP-2), publicly released GR00T-N1-2B, 2.2 billion (R). Names don't say which is stronger.
Evidence: O official · R research · M media
| R | 256 bins | OpenVLA's action head: each action dimension is discretised into 256 bins and predicted as tokens |
| R | Flow matching | π-zero's action head: on a pre-trained vision-language model, flow matching (a variant of diffusion) generates continuous actions |
| R | 50 hertz | With action chunking plus flow matching, π-zero runs control at up to 50 hertz (per the paper; peak, for dexterous tasks like folding laundry) |
| R | Fast + slow systems | GR00T N1 as a whole is a dual system: the slow System 2 (vision-language module) understands, and the fast System 1 (diffusion transformer action module) outputs actions in real time |
| R | 120 hertz | GR00T N1's fast system (a diffusion transformer) outputs closed-loop actions at up to 120 hertz (paper) |
| R | Flow matching | GR00T N1's fast system (a diffusion transformer, DiT) is also trained with flow matching to generate actions (paper) |
| R | Llama 2 | OpenVLA's language backbone is Llama 2; its vision encoder fuses DINOv2 and SigLIP features |
| R | 7 billion | parameters in OpenVLA (7B); trained on about 970,000 real robot demonstrations (paper) |
| R | PaliGemma | is π-zero's vision-language backbone; its weights are initialised from this open model (paper) |
| R | 3.3 billion | π-zero has about 3.3 billion parameters in total (a 3 billion VLM + a 300 million action expert, paper) |
| R | Eagle-2 | GR00T N1 uses NVIDIA's Eagle-2 as its vision-language backbone (paper) |
| R | SmolLM2 + SigLIP-2 | Eagle-2 is fine-tuned from the SmolLM2 language model and the SigLIP-2 image encoder (paper) |
| R | 2.2 billion | GR00T-N1-2B has about 2.2 billion parameters in total, about 1.34 billion of them in the vision-language backbone (paper) |
Further reading
- OpenVLA: An Open-Source Vision-Language-Action Model (arXiv 2406.09246)
https://arxiv.org/abs/2406.09246 - π0: A Vision-Language-Action Flow Model for General Robot Control (arXiv 2410.24164)
https://arxiv.org/abs/2410.24164 - GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (arXiv 2503.14734)
https://arxiv.org/abs/2503.14734