Physical AI Lab // Episode 3.2 // One-pager
World models Making data with Cosmos
Predicting the future (Predict) and transferring style (Transfer)
Three key points
- A world model is a digital twin of the world: give it past frames plus an action and it predicts the next moment; it is trained on oceans of video (about 20 million hours, 100 million clips, R).
- Cosmos Predict generates video going forward (diffusion 7B/14B plus autoregressive; trained on ten thousand H100s for three months, R; 78.3% human preference in robot instruction prediction, R). The paper lists evaluation, planning and policy training as possible uses, but gives no empirical results for them yet.
- Cosmos Transfer does conditional generation from segmentation/depth/edges, world-to-world transfer and sim2real (real time on a top-end rack, R); but looking real is not correct physics.
Evidence: O official · R research · M media
| R | 20 million hours | of raw video used to pre-train Cosmos, cut into about 100 million clips (NVIDIA) |
| R | 7 billion / 14 billion | parameters, the two sizes of the Cosmos-Predict1 diffusion world model (plus autoregressive 4 billion and 12 billion, NVIDIA) |
| R | 10,000 H100s | in the GPU cluster that trained all the paper's world foundation models, over three months (NVIDIA) |
| R | 78.3% | overall human-rater preference for the fine-tuned Cosmos-Predict model in robot instruction video prediction (baseline 13.0%, NVIDIA) |
| R | Segmentation · depth · edges | are the spatial control inputs Cosmos-Transfer uses to generate worlds (NVIDIA) |
| R | Real time (top-end rack) | world generation on one NVIDIA GB200 NVL72 rack (NVIDIA, peak configuration) |
Further reading
- Cosmos World Foundation Model Platform (arXiv 2501.03575)
https://arxiv.org/abs/2501.03575 - Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control (arXiv 2503.14492)
https://arxiv.org/abs/2503.14492 - NVIDIA Cosmos (open-source repo and weights)
https://github.com/nvidia-cosmos/cosmos-predict1