Physical AI Lab // Episode 3.2 // One-pager

World models Making data with Cosmos

Predicting the future (Predict) and transferring style (Transfer)

Three key points

  1. A world model is a digital twin of the world: give it past frames plus an action and it predicts the next moment; it is trained on oceans of video (about 20 million hours, 100 million clips, R).
  2. Cosmos Predict generates video going forward (diffusion 7B/14B plus autoregressive; trained on ten thousand H100s for three months, R; 78.3% human preference in robot instruction prediction, R). The paper lists evaluation, planning and policy training as possible uses, but gives no empirical results for them yet.
  3. Cosmos Transfer does conditional generation from segmentation/depth/edges, world-to-world transfer and sim2real (real time on a top-end rack, R); but looking real is not correct physics.

Evidence: O official · R research · M media

R20 million hoursof raw video used to pre-train Cosmos, cut into about 100 million clips (NVIDIA)
R7 billion / 14 billionparameters, the two sizes of the Cosmos-Predict1 diffusion world model (plus autoregressive 4 billion and 12 billion, NVIDIA)
R10,000 H100sin the GPU cluster that trained all the paper's world foundation models, over three months (NVIDIA)
R78.3%overall human-rater preference for the fine-tuned Cosmos-Predict model in robot instruction video prediction (baseline 13.0%, NVIDIA)
RSegmentation · depth · edgesare the spatial control inputs Cosmos-Transfer uses to generate worlds (NVIDIA)
RReal time (top-end rack)world generation on one NVIDIA GB200 NVL72 rack (NVIDIA, peak configuration)

Further reading