Physical AI Lab // Episode 2.3 // One-pager

Make it understand a sentence

Fine-tuning SmolVLA: from language instructions to actions

Three key points

A VLA (vision-language-action model) turns the image, the robot's state and one language instruction together into actions, so it can try a task even when you phrase it differently.
SmolVLA is compact (about 450 million parameters, O) and can be fine-tuned from the base on a small GPU; the paper says it performs on par with models about ten times larger (R).
Community-data pre-training raises the SO-100 success rate to 78.3% (O, team experiments), and the team says asynchronous inference responds faster and copes better with disturbances; but on objects it has never seen, generalization still depends on data.

O	450 million	parameters in SmolVLA's base model smolvla_base; compact, and runs on consumer hardware (Hugging Face official blog)
R	10 times	larger VLA models; the SmolVLA paper says that, despite its compact size, it performs on par with them
O	30% faster	responses with asynchronous inference, and tasks finished per unit of time double (SmolVLA team, real-robot experiments)
O	78.3%	success rate on SO-100 after community-data pre-training reaches 78.3%, about 26.6 percentage points higher than without (SmolVLA team experiments)
O	about 50 demos	recommended to record before fine-tuning SmolVLA; the docs say a 25-episode dataset was not enough and performed badly (LeRobot official docs)
O	20,000 steps · about 4 hours	fine-tuning smolvla_base for about 20,000 steps takes about 4 hours on a single A100 (LeRobot official docs)