Physical AI Lab // Episode 2.6 // One-pager

To the edge: quantization, latency, control rate

A robot can only correct itself as fast as its model runs

Three key points

  1. Latency sets the upper limit on control rate: highest rate = 1 / single-inference latency. One paper says 1 kHz is a necessity for its real-time control and the minimum frequency required by safety standards (R); one large model was measured re-planning about five times a second (O, Amazon, H100).
  2. Quantization's surest gain is size: int8 shrinks a model to about a quarter (O, PyTorch); speed-ups depend on hardware and model and must be measured (in this episode's notebook, the quantized model was slower on a CPU).
  3. Pair slow planning with fast control, decoupled by action chunking; an edge module can run the latest models within about a hundred and thirty watts (O, NVIDIA). Latency can only be managed, never eliminated.

Evidence: O official · R research · M media

R1 kHzthe paper says sustaining 1 kHz is a necessity for real-time control and the minimum frequency required by safety standards (for its own Franka multi-arm real-time control framework)
O≈5 Hzmeasured by Amazon: on an H100 with 1 ODE step, π0 generates a chunk in under 200 ms, so it can re-plan about 5 times a second (one model, one hardware setup)
O4×smaller — int8 quantization shrinks a model to about a quarter of its size (PyTorch blog; speed-ups are separate and depend on hardware and model)
O2–3×the official reference speed-up for int8 inference; the real figure depends on hardware, backend and model (PyTorch blog)
O130 Wis enough for a robot's edge computing module to run the latest AI models; peak 2,070 trillion operations/s (NVIDIA, FP4)

Further reading