Physical AI Lab // Episode 2.6 // One-pager
To the edge: quantization, latency, control rate
A robot can only correct itself as fast as its model runs
Three key points
- Latency sets the upper limit on control rate: highest rate = 1 / single-inference latency. One paper says 1 kHz is a necessity for its real-time control and the minimum frequency required by safety standards (R); one large model was measured re-planning about five times a second (O, Amazon, H100).
- Quantization's surest gain is size: int8 shrinks a model to about a quarter (O, PyTorch); speed-ups depend on hardware and model and must be measured (in this episode's notebook, the quantized model was slower on a CPU).
- Pair slow planning with fast control, decoupled by action chunking; an edge module can run the latest models within about a hundred and thirty watts (O, NVIDIA). Latency can only be managed, never eliminated.
Evidence: O official · R research · M media
| R | 1 kHz | the paper says sustaining 1 kHz is a necessity for real-time control and the minimum frequency required by safety standards (for its own Franka multi-arm real-time control framework) |
| O | ≈5 Hz | measured by Amazon: on an H100 with 1 ODE step, π0 generates a chunk in under 200 ms, so it can re-plan about 5 times a second (one model, one hardware setup) |
| O | 4× | smaller — int8 quantization shrinks a model to about a quarter of its size (PyTorch blog; speed-ups are separate and depend on hardware and model) |
| O | 2–3× | the official reference speed-up for int8 inference; the real figure depends on hardware, backend and model (PyTorch blog) |
| O | 130 W | is enough for a robot's edge computing module to run the latest AI models; peak 2,070 trillion operations/s (NVIDIA, FP4) |
Further reading
- PyTorch · Introduction to Quantization on PyTorch
https://pytorch.org/blog/introduction-to-quantization-on-pytorch/ - multipanda_ros2: 1 kHz real-time multi-arm control (arXiv 2602.02269)
https://arxiv.org/abs/2602.02269 - AWS · Fine-Tuning π0 on SageMaker HyperPod EKS (re-planning at about 5 Hz)
https://aws.amazon.com/blogs/physical-ai/fine-tuning-%CF%800-pi-zero-for-robotic-manipulation-on-amazon-sagemaker-hyperpod-eks/ - NVIDIA · Jetson Thor Now Available (2025-08-25)
https://nvidianews.nvidia.com/news/nvidia-blackwell-powered-jetson-thor-now-available-accelerating-the-age-of-general-robotics