World Models and Planning

A world model predicts what happens next given an action. Combine that with a planner and you get an agent that can reason about consequences before acting.

The idea

A world model is a learned model of how the environment evolves. Given current state and an action, predict next state. Plan by simulating actions in the world model, pick the action sequence that leads to the best outcome. World models are an old idea (Schmidhuber, 1990s) finding new traction in robotics and embodied AI; the bet is that learned simulation enables planning that pure RL can't.

The "model-free vs model-based" framing. Model-free RL: learn a policy that maps state to action. Model-based RL: learn a model of the world, plan within the model. Model-based has theoretical advantages (sample efficiency, transfer); has been historically harder to make work in complex domains.

The reason for renewed interest. 2024-2026 advances in generative models (vision, video) make learning realistic world models more tractable. Neural networks can now predict next-frame visual states with surprising fidelity. Combined with planning algorithms, this enables new capabilities.

The honest framing. World models work for narrow domains; struggle with open-world complexity. Production deployments are rare. The technology is real but young; expect maturity in 2027-2028 for specific applications, broader adoption later.

Research lines

Multiple active research directions:

• Dreamer (DeepMind), RL agent with explicit world model; reaches strong performance on Atari and physical control with relatively few samples.
• Diffusion world models, generative video models adapted as world models. Realistic visual prediction; integrates with high-resolution observations.
• Genie (DeepMind), learns interactive world from video alone; opens door to data-efficient world model training.
• JEPA-based world models (Meta), Yann LeCun's research direction; world models in latent space rather than pixel space.

The Dreamer line. Trains a world model jointly with a policy. The policy is trained by rollouts in the imagined world. Sample efficiency is the headline benefit, Dreamer reaches strong Atari performance with 100K samples vs millions for pure RL.

The diffusion world models. Use diffusion models (the same architecture as image generators) to predict next visual frames given action. Production-quality video; expensive to roll out for many candidate actions. Useful when high-fidelity simulation matters.

The Genie line. Learns playable world models from video alone (no action labels). User can input actions; model predicts next frames. Demonstrates that data-efficient world modeling from massive video data is possible. Pure research; commercial applications years out.

The JEPA-based line. World models in latent space. Cheaper than pixel-space prediction; arguably more aligned with how brains might do it. LeCun's bet on the right direction; results are improving but pixel-space methods are currently more mature.

The convergence question. Will future world models be pixel-space or latent-space? Generative or predictive? The research lines run in parallel; convergence likely as approaches synthesise. By 2028, expect a clearer winner.

Why it matters

For embodied AI, world models enable planning rather than reaction. Robot can simulate "if I push this object, will it tip over?" before doing it. Autonomous vehicles can simulate "if I change lanes now, will the gap close?". The capability shift from reactive to anticipatory is significant; products that need anticipation can't avoid world models.

The robotics use case. Pure reactive policies struggle with multi-step planning. World models enable explicit planning; the robot considers consequences before acting. Sample efficiency improves; interpretability improves; safety improves (the robot can detect "this plan looks dangerous in simulation; pick a different one").

The autonomous-vehicle use case. Self-driving systems combine reactive perception with planning. World models that predict other vehicles' behaviour enable better planning. Currently mostly bespoke architectures; world-model-style learning may improve with future generations.

The simulation-acceleration use case. Replace expensive physics simulations with neural-network surrogates that are 100-1000x faster. Climate modeling, materials science, drug discovery all benefit. The "world model" framing extends to scientific computing.

The game AI use case. Game AI traditionally hand-coded. World-model approaches let game AI plan in learned models of the game world. Better adaptation to player behaviour; richer NPC interactions; emerging in 2026 production games.

The "general intelligence" framing. Some researchers see world models as essential for general intelligence, the ability to imagine and plan. Whether this framing is correct is philosophical; whether world models help with current AI capability is empirical and the empirical answer is yes for many domains.

Common antipatterns

Treating world models as a panacea. Domain-specific; not universal. Match to use case.

Pure pixel-space world models for high-resolution. Compute cost is prohibitive. Use latent space when resolution matters.

Skipping uncertainty quantification. Plans based on overconfident world models fail. Track model uncertainty; use uncertainty-aware planning.

One world model for everything. Different scales (object dynamics, social dynamics, large-scale) need different models. Hierarchical structure helps.

What to do this week

Three moves. (1) For embodied or planning-relevant applications, evaluate whether world-model approaches fit. The sample-efficiency gain is the headline benefit. (2) Read Dreamer paper or watch the talk. The intuition for model-based RL is foundational for thinking about world models. (3) If your domain has expensive simulations (physics, climate, biology), evaluate neural-surrogate alternatives. The 100x speedup unlocks workflows that weren't feasible.