A curated list of world models for video generation, interactive simulation, and 3D world creation.
From video synthesis to playable game worlds to explorable 3D environments.
- What Are World Models?
- Surveys
- Foundation World Models
- World Models for Video Generation
- World Models for Game Simulation
- World Models for 3D World Generation
- World Models for Creative & Entertainment Applications
- Key Approaches & Architectures
- Benchmarks & Evaluation
- Theory, Positions & Explainability
- Key Blog Posts & Technical Reports
- Contributing
- Citation
A world model is an AI system that learns to simulate how the world works, capturing physics, dynamics, and cause-and-effect — to predict what happens next given an action or input. Two foundational works define the field:
- World Models (Ha & Schmidhuber, 2018) — Train agents inside learned environment simulators (VAE + RNN + controller). [Paper] [Website]
- A Path Towards Autonomous Machine Intelligence (LeCun, 2022) — Proposed JEPA: world models that predict in latent space, not pixel space. [Paper]
For video generation and interactive simulation, world models go beyond passive synthesis — they learn to produce physically plausible environments that respond to user actions in real time.
New to the field? Read The Evolution of World Models for Media, a deep analysis of how methods build upon each other — from Ha & Schmidhuber (2018) to real-time multiplayer world simulation (2026).
- Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond (May 2024) — Comprehensive taxonomy of world models across video, driving, and agents toward AGI. [Paper] [Code]
- From Efficient Multimodal Models to World Models: A Survey (Jul 2024) — Charts evolution from multimodal large models toward world models via 3D generation and embodied intelligence. [Paper]
- Understanding World or Predicting Future? A Comprehensive Survey of World Models (Nov 2024) — Distinguishes world models by dual functions: understanding mechanisms versus predicting future states. [Paper] [Code]
- Four Principles for Physically Interpretable World Models (Mar 2025) — Proposes four principles for world models with verifiable, physically grounded latent representations. [Paper]
- World Models in Artificial Intelligence: Sensing, Learning, and Reasoning Like a Child (Mar 2025) — Advocates child-like cognitive development integrating six research areas for genuine AI reasoning. [Paper]
- Exploring the Evolution of Physics Cognition in Video Generation: A Survey (Mar 2025) — Three-tier taxonomy of video generation's evolution from visual mimicry to physical comprehension. [Paper] [Code]
- 3D and 4D World Modeling: A Survey (Sep 2025) — First comprehensive review dedicated to 3D/4D world modeling with standardized taxonomy and datasets. [Paper] [Code]
- From Masks to Worlds: A Hitchhiker's Guide to World Models (Oct 2025) — Traces the developmental path from masked learning through unified architectures to interactive generation. [Paper]
- Simulating the Visual World with Artificial Intelligence: A Roadmap (Nov 2025) — Conceptualizes video models as implicit world simulators plus renderers across four technology generations. [Paper] [Website]
- A Mechanistic View on Video Generation as World Models: State and Dynamics (Jan 2026) — Disentangles state construction from dynamics modeling; advocates functional benchmarks over visual quality. [Paper]
- The Trinity of Consistency as a Defining Principle for General World Models (Feb 2026) — Unified modal-spatial-temporal consistency framework with CoW-Bench evaluation protocol. [Paper]
Large-scale, general-purpose world models that serve as platforms for multiple downstream applications.
- Sora (OpenAI, Feb 2024) — Spacetime patches in a Diffusion Transformer (DiT); emergent 3D consistency and physics from text-to-video at scale. [Technical Report]
- Genie (DeepMind, Feb 2024) — Learns action control from unlabeled video; 11B-param model discovers latent actions without any action annotations. [Paper] [Blog]
- V-JEPA (Meta, Feb 2024) — Non-generative JEPA; predicts masked video in latent space (not pixels), 1.5-6x training efficiency with frozen encoder transfer. [Blog] [Code]
- Veo (Google DeepMind, May 2024) — 1080p minute-long text-to-video with cinematic camera control; Google's flagship video model and the bar Veo 2/3 build on. [Blog] [Website]
- 1X World Model (1X Technologies, Sep 2024) — Action-conditioned humanoid forecaster predicts full-body humanoid futures from video for evaluating policies in "bits, not atoms". [Blog] [Tech Report]
- Genie 2 (DeepMind, Dec 2024) — Autoregressive latent diffusion for 3D worlds from a single image; minute-long consistency with emergent NPC behavior. [Blog]
- Veo 2 (Google DeepMind, Dec 2024) — 4K resolution + improved physics; stronger object permanence and motion realism, exceeded Sora benchmarks at launch. [Blog] [Website]
- Genesis (Multi-institution collab, Dec 2024) — 43M FPS generative physics engine; rigid + MPM + SPH + FEM + PBD fluid solvers in one Pythonic API, generates 4D worlds from natural language. [Website] [Code]
- Cosmos (NVIDIA, Jan 2025) — Open-source platform (4B-14B params); full pipeline with tokenizers, pre-trained models, permissive license. [Paper] [Website] [Code]
- Veo 3 (Google DeepMind, May 2025) — Native synchronized audio; first major text-to-video to jointly generate dialogue, music, and ambient sound aligned with frames in one pass. [Blog] [Website]
- V-JEPA 2 (Meta, Jun 2025) — 1M+ hours pre-training, <62 hours fine-tuning for zero-shot robotic planning; 16x faster than video-generation-based planners. [Paper] [Website] [Code]
- V-JEPA 2.1
(Meta, Mar 2026) — Dense Predictive Loss + Deep Self-Supervision across multiple encoder layers; multi-modal tokenizers unlock spatially organized, temporally stable video representations. [Paper]
- Genie 3 (DeepMind, Aug 2025) — First real-time interactive world model (24fps, 720p); promptable world events via text during live play. [Blog]
- Emu3.5 (BAAI, Oct 2025) — Discrete Diffusion Adaptation (DiDA) converts autoregressive decoding to parallel prediction for ~20x speedup; 10T+ token training. [Paper] [Website] [Code]
- RTFM (World Labs, Oct 2025) — Learned renderer with spatial memory and context juggling for unbounded persistence; single H100. [Blog]
- Odyssey-2 (Odyssey, Oct 2025) — 20 FPS streaming interactive video via multi-stage causal training; first general-purpose model to imagine interactive video instantly from text prompts. [Blog]
- Cosmos-Predict2.5 (NVIDIA, Oct 2025) — Unified flow architecture for Text/Image/Video2World in one model; RL-based post-training on 200M clips. [Paper] [Code]
- Veo 3.1 (Google DeepMind, Oct 2025) — Narrative control + clip extension; preserves subject, outfit, and camera continuity when extending clips beyond their original length. [Blog] [Website]
- PAN (MBZUAI, Nov 2025) — Generative Latent Prediction: LLM dynamics backbone + diffusion decoder; text-grounded open-domain long-horizon simulation. [Paper] [Website]
- Marble (World Labs, Nov 2025) — Chisel decouples 3D structure from style; multimodal input to exportable Gaussian splats, meshes, or video. [Blog]
- GWM-1 (Runway, Dec 2025) — Cross-domain action conditioning (camera, events, poses, speech); three variants: worlds, avatars, robotics. [Blog]
- LingBot-World (Robbyant, Jan 2026) — Sub-1s latency, 10-min stable generation; 28B open-source model, internet video + Unreal Engine data. [Paper] [Website] [Code]
- Lucy 2.0 (Decart, Jan 2026) — Sub-100ms real-time world transformation; 30 fps 1080p live video editing via mega-kernel optimized diffusion with smart history augmentation against drift. [Blog] [Website]
- Happy Oyster
- Odyssey-2 Max
- WorldDreamer (Jan 2024) — Masked visual token prediction for multi-domain world modeling; jointly models appearance and dynamics without domain-specific design. [Paper] [Code]
- LWM (Feb 2024) — RingAttention scales context from 4K to 1M tokens; enables coherent world modeling over million-length video-language sequences. [Paper] [Code]
- Pandora (Jun 2024) — Hybrid LLM (7B) + video model; fuses pretrained language model with video generator for text-action controlled simulation. [Paper] [Code]
- Owl-1 (Dec 2024) — Latent state variable decoding; prevents long-term inconsistency accumulation by decoding observations from abstract state representations. [Paper]
- Aether (Mar 2025) — Unified task-interleaved learning combining 4D reconstruction, prediction, and geometry-informed planning in one model. [Paper] [Website]
- DeepVerse (Jun 2025) — Explicit geometric predictions from previous timesteps incorporated into action-conditioned 4D video generation. [Paper]
- UnityVideo (Dec 2025) — Dynamic noising + modality switcher unifies heterogeneous modalities for cross-modal world-aware video generation. [Paper] [Website] [Code]
- NuiWorld (Jan 2026) — Generative bootstrapping synthesizes training data; compresses variable scene chunks to flattened vectors for scalable generation. [Paper]
- DreamWorld
- LiveWorld
- Olaf-World (Feb 2026) — Seq-delta-REPA aligns latent actions to temporal semantic effects for cross-context transferability. [Paper] [Website] [Code]
- iVideoGPT (May 2024) — Compressive tokenization discretizes high-dimensional observations for scalable GPT-style interactive world modeling. [Paper] [Website] [Code]
- AdaWorld (HKUST/Harvard, Mar 2025) — Self-supervised latent action extraction from video enables zero-shot action transfer across unseen environments. [Paper] [Website] [Code]
- Vid2World (May 2025) — Causalizes pretrained video diffusion models by reshaping architecture and training objectives into interactive simulators. [Paper] [Website]
- VRAG (May 2025) — Video retrieval augmented generation with explicit global state conditioning reduces long-term compounding errors. [Paper]
- Astra (Dec 2025) — Noise-augmented history memory balances responsiveness with temporal coherence in streaming autoregressive denoising. [Paper] [Website] [Code]
- RELIC (Adobe, Dec 2025) — Camera-aware compressed latent tokens in KV cache enable real-time 16 FPS long-horizon interactive generation. [Paper] [Website]
- LIVE (Feb 2026) — Cycle-consistency with forward-backward generation eliminates need for teacher distillation in long-horizon interaction. [Paper] [Website]
- MosaicMem
- WorldCache
- HyDRA
- Generated Reality (Stanford, Feb 2026) — Conditions on tracked 3D head pose + joint-level hand poses for dexterous egocentric world simulation. [Paper]
- GEN3C (NVIDIA, Mar 2025) — 3D cache conditions video diffusion on 2D renderings for precise camera control and multi-view consistency.
CVPR 2025[Paper] - FAR (NUS, Mar 2025) — Asymmetric patchify kernels apply large kernels to distant frames for efficient long-context autoregressive video. [Paper] [Code] [Website]
- Long-Context SSM World Models (Stanford/Adobe, May 2025) — Hybrid SSM + local attention; block-wise scanning trades spatial detail for extended temporal memory at constant cost. [Paper] [Website]
- Yume (Shanghai AI Lab, Jul 2025) — Masked Video Diffusion Transformer with memory enables infinite autoregressive world generation. [Paper] [Website] [Code]
- LongVie 2 (Dec 2025) — Three-stage training with degradation-aware bridging and history-context guidance for 5-minute controllable video. [Paper] [Website]
- Yume-1.5 (Shanghai AI Lab, Dec 2025) — Unified context compression via linear attention + bidirectional distillation for real-time streaming interaction. [Paper] [Website] [Code]
- UCM
- Stereo World Model
- VerseCrafter (Jan 2026) — 4D geometric control via static point cloud background + per-object 3D Gaussian trajectories. [Paper] [Website]
- How Far is Video Generation from World Model (ByteDance, Nov 2024) — 2D simulation testbed reveals models exhibit case-based rather than rule-based physical reasoning.
ICML 2025[Paper] [Website] [Code] - Geometry Forcing (Microsoft, Jul 2025) — Angular and scale alignment guide video diffusion models to internalize latent 3D geometric representations.
ICLR 2026[Paper] [Website] - ProPhy (Dec 2025) — Mixture of semantic + refinement physics experts progressively align token-level physical dynamics during generation. [Paper]
- World Models That Know When They Don't Know (Princeton, Dec 2025) — C3 framework for continuous-scale calibrated uncertainty at subpatch-level localization. [Paper]
- Interpreting Physics in Video World Models
- Inference-time Physics Alignment (Meta FAIR, Jan 2026) — WMReward uses V-JEPA 2 as reward model for inference-time denoising trajectory search. [Paper]
- LaWM
- DIAMOND (May 2024) — Diffusion replaces discrete latents for world modeling; preserves visual details that matter for RL, SoTA on Atari. [Paper] [Code]
- GameNGen (Google, Aug 2024) — First neural game engine; autoregressive diffusion with conditioning augmentation runs DOOM at interactive frame rates. [Paper]
- Oasis (Decart/Etched, Oct 2024) — Dynamic noising schedule during inference reduces error accumulation; transformer-based real-time Minecraft-style generation. [Website]
- GameFactory (Kuaishou, Jan 2025) — Domain adapter decouples game style from action control for scene-generalizable novel game creation. [Paper] [Website] [Code]
- AnimeGamer (Tencent, Apr 2025) — MLLM generates game states with action-aware multimodal representations for infinite anime life simulation. [Paper] [Website]
- MineWorld (Microsoft, Apr 2025) — Parallel decoding predicts spatially redundant tokens simultaneously for 4-7 FPS real-time Minecraft simulation. [Paper] [Website]
- WORLDMEM (Apr 2025) — Memory bank with attention maintains long-term 3D spatial consistency across viewpoint changes in game sessions. [Paper] [Website] [Code]
- Matrix-Game (Skywork, Jun 2025) — Two-stage pipeline: unlabeled pretraining for environment understanding, then action-labeled training for control. [Paper] [Code]
- RealPlay (Jun 2025) — Iterative chunk-wise prediction enables transfer from virtual game controls to photorealistic real-world scenarios. [Paper] [Website] [Code]
- Matrix-Game 2.0 (Skywork, Aug 2025) — Few-step causal diffusion + action injection enables streaming 25 FPS interactive generation. [Paper] [Website]
- Hunyuan-GameCraft-2 (Tencent, Nov 2025) — Text-driven interaction injection enables natural language control instead of fixed keyboard/mouse inputs. [Paper] [Website]
- Captain Safari (Nov 2025) — Pose-conditioned retriever fetches pose-aligned world tokens from persistent memory for open-ended exploration. [Paper] [Website]
- SIMA 2 (DeepMind, Dec 2025) — Autonomous self-improvement through task/reward generation from foundation model; conversational generalist game agent. [Paper]
- Waypoint-1 (Overworld, Jan 2026) — Self-forcing trained from scratch on 10K hours gameplay for real-time interactive world generation. [Blog]
- Solaris (Feb 2026) — First multiplayer world model; synchronized capture with Checkpointed Self Forcing for multi-agent view consistency in Minecraft. [Paper] [Website]
- Matrix-Game 3.0
- SCOPE
- Agora-1
- HunyuanWorld 1.0 (Tencent, Jul 2025) — Semantically layered 3D mesh with 360-degree panoramic proxies for immersive, decomposable world generation. [Paper] [Website] [Code]
- Matrix-3D (Aug 2025) — Trajectory-guided panoramic video diffusion conditioned on scene mesh renders for 360-degree geometric consistency. [Paper] [Website]
- WorldGen (Meta, Nov 2025) — LLM-driven procedural planning with navmesh-conditioned reconstruction for traversable compositional 3D scenes. [Blog]
- HY-World 2.0
- FantasyWorld (Alibaba, Sep 2025) — Frozen video model + trainable geometric branch with cross-branch supervision for geometry-consistent world modeling. [Paper]
- EvoWorld (JHU, Oct 2025) — Evolving explicit 3D memory guides video generation through geometric reprojections for panoramic consistency. [Paper] [Code]
- Terra (Tsinghua/Kuaishou, Oct 2025) — Point-to-Gaussian VAE operates in intrinsic 3D latent space for native geometric consistency. [Paper] [Website]
- WorldGrow (Oct 2025) — Hierarchical 3D block inpainting with coarse-to-fine strategy for infinite scene expansion. [Paper] [Code]
- TRELLISWorld (CMU/HKUST, Oct 2025) — Training-free multi-tile denoising repurposes object diffusion models for scalable scene generation. [Paper]
- GeoWorld
- MagicWorld (NUS/Zhejiang, Nov 2025) — Action-guided 3D geometry module constructs point clouds for viewpoint-stable video exploration. [Paper]
- ChronosObserver (Beihang, Dec 2025) — Training-free hyperspace synchronizes multi-view diffusion sampling trajectories for 4D consistency. [Paper]
- DynamicVerse (Xiamen/Meta, Dec 2025) — Window-based bundle adjustment with global optimization for metric-scale 4D annotations. [Paper]
- SeeU (Purdue/Samsung, Dec 2025) — 2D→4D→2D framework reconstructs continuous spatiotemporal dynamics from single views. [Paper] [Website]
- Visionary (Shanghai AI Lab, Dec 2025) — WebGPU platform unifies ONNX inference and 3DGS rendering per-frame for browser-based world interaction. [Paper] [Website]
- WonderZoom (Stanford, Dec 2025) — Scale-adaptive Gaussian surfels enable multi-scale zoom with progressive detail synthesis. [Paper] [Website]
- WorldPlay (HKUST/Tencent, Dec 2025) — Reconstituted context memory dynamically rebuilds past frames with temporal reframing for geometric consistency. [Paper]
- TeleWorld (TeleAI, Dec 2025) — Generation-reconstruction-guidance paradigm with dynamic 4D representation for real-time multimodal interaction. [Paper]
- NeoVerse (CAS/CreateAI, Jan 2026) — Pose-free feed-forward 4D reconstruction with online monocular degradation simulation from in-the-wild video. [Paper] [Website]
- Beyond Pixel Histories
- VEGA-3D
- From Part to Whole
ICME 2026[Paper] - Lyra 2.0
- Spark 2.0
- Martian World Models (NVIDIA/UT Austin, Jul 2025) — Metric-accurate 3D reconstruction from NASA stereo nav images for controllable Mars terrain synthesis. [Paper] [Website]
- LatticeWorld (NetEase/Beihang, Sep 2025) — LLM-generated symbolic layouts rendered in Unreal Engine 5 for 90x faster interactive multi-agent world production. [Paper]
- ChronoEdit (NVIDIA, Oct 2025) — Temporal reasoning tokens treat image editing as video generation; tokens dropped after denoising for efficiency. [Paper] [Website]
- MorphoSim (UCSC/UCLA, Oct 2025) — Feature field distillation enables object-level 4D editing without full scene regeneration. [Paper] [Code]
- Inferix (Alibaba/HKUST, Nov 2025) — Semi-autoregressive block diffusion with LLM-style KV cache for efficient variable-length video generation. [Paper] [Code]
- WorldWander (NUS, Nov 2025) — Collaborative position encoding with in-context perspective alignment for egocentric-exocentric video translation. [Paper] [Code]
- AVWM (Tsinghua, Dec 2025) — Modality-expert diffusion transformer for synchronized audio-visual generation with binaural spatial cues. [Paper]
- WorldPack (UTokyo/DeepMind, Dec 2025) — Hierarchical trajectory packing compresses long-horizon memory for spatial consistency in world modeling. [Paper]
- IC-World (NTU/Tencent, Dec 2025) — GRPO-based geometry rewards enforce multi-view consistency in in-context shared world generation. [Paper] [Code]
- Walk through Paintings
- Choreographing a World of Dynamic Objects (Stanford, Jan 2026) — Rectified flow SDS with hierarchical Lagrangian deformations for category-agnostic 4D motion synthesis. [Paper] [Website]
How the model produces world states frame by frame.
| Paradigm | How It Works | Strengths | Limitations | Representative Works |
|---|---|---|---|---|
| Autoregressive (Token) | Predict next visual token sequentially via transformer | Scalable; leverages LLM-era infrastructure | Compounding error over long horizons | Genie, Oasis, Matrix-Game |
| Autoregressive (Diffusion) | Denoise each frame conditioned on previous frames causally | High visual fidelity per frame | Slower inference per step | GameNGen, DIAMOND, Astra |
| Flow-based | Unified flow matching for continuous generation | Single model handles multiple input modalities | Training complexity | Cosmos-Predict2.5 |
| Latent Prediction (JEPA) | Predict in abstract representation space, not pixels | Sample-efficient; avoids pixel-level hallucination | Cannot directly render output | V-JEPA, V-JEPA 2, AdaWorld |
| Hybrid (LLM + Diffusion) | LLM reasons about dynamics in latent space; diffusion renders | Combines semantic reasoning with visual quality | Two-stage inference overhead | PAN, Pandora |
How users interact with and steer the generated world.
| Method | Mechanism | Representative Works |
|---|---|---|
| Latent action discovery | Learn action spaces from unlabeled video; no annotations needed | Genie, AdaWorld, Olaf-World |
| Keyboard/mouse action injection | Condition on explicit discrete inputs per frame | GameNGen, Oasis, Matrix-Game 2.0 |
| Natural language control | Text instructions drive world dynamics and events | Genie 3, Hunyuan-GameCraft-2, Yume-1.5 |
| Camera/pose conditioning | 3D camera trajectories or body pose control viewpoint | GEN3C, Generated Reality, Captain Safari |
| Cross-domain action conditioning | Unified interface across camera, events, poses, speech | GWM-1 |
How models maintain coherence over extended generation.
| Strategy | How It Works | Representative Works |
|---|---|---|
| Spatial memory bank | Store and retrieve world tokens indexed by 3D position | WORLDMEM, RELIC, Captain Safari |
| KV cache compression | Compress history into latent tokens in the attention cache | RELIC, Yume-1.5 |
| Context juggling | Dynamically select relevant past frames from spatial memory | RTFM |
| Reconstituted context | Rebuild past frames with temporal reframing on the fly | WorldPlay |
| Retrieval augmented generation | Retrieve relevant past video segments to condition generation | VRAG |
| Hybrid SSM + Attention | SSM for long-range temporal; local attention for spatial detail | Long-Context SSM WMs |
| Cycle-consistency | Forward-backward generation enforces temporal coherence | LIVE |
| Checkpointed self-forcing | Periodic ground-truth resets during autoregressive rollout | Solaris, Waypoint-1 |
How models incorporate spatial understanding.
| Representation | Trade-off | Representative Works |
|---|---|---|
| Implicit (pixel-space) | No explicit 3D; fast but limited viewpoint consistency | GameNGen, Oasis, DIAMOND |
| 3D cache / depth conditioning | Lightweight 3D signal guides 2D generation | GEN3C, Geometry Forcing |
| Point cloud / point latents | Native 3D latent space; efficient but sparse | Terra, MagicWorld, EvoWorld |
| Gaussian splatting | Real-time renderable; exportable 3D assets | Marble, Visionary, HunyuanWorld |
| Mesh-based | Traditional 3D; compatible with game engines | HunyuanWorld, Matrix-3D, WorldGen |
| 4D (dynamic 3D) | Spatiotemporal; captures motion and deformation | NeoVerse, DynamicVerse, SeeU |
- WorldSimBench (Oct 2024) — Dual perceptual + manipulative evaluation; tests whether generated videos translate into valid control signals across embodied scenarios. [Paper] [Website]
- WorldModelBench (Feb 2025) — Subtle physics violation detection; 67K human labels measuring whether models catch mass conservation breaches and irregular object changes. [Paper] [Website]
- VideoVerse (Oct 2025) — Event-level temporal causality; 300 prompts with 815 events evaluating causal reasoning depth beyond per-frame quality. [Paper]
- Gen-ViRe (Nov 2025) — Chain-of-Frames reasoning; decomposes visual reasoning into 6 cognitive dimensions and 24 subtasks measuring multi-step planning. [Paper] [Code]
- 4DWorldBench (Nov 2025) — Unified 3D/4D evaluation; maps diverse input modalities into textual representations for perceptual quality, physical realism, and 4D consistency. [Paper]
- SmallWorlds (Nov 2025) — Isolated dynamics testbed; compares RSSM, Transformers, Diffusion, and Neural ODEs under controlled rollout degradation. [Paper]
- WorldBench (Jan 2026) — Disentangled physics diagnostics; isolates individual physical principles (object permanence, friction, viscosity) instead of testing them jointly. [Paper] [Website]
- PhysicsMind (Jan 2026) — Law-consistent reasoning + generation; evaluates both VQA physical reasoning and physics-compliant video generation across canonical laws. [Paper]
- Wow, wo, val!
- MIND (Feb 2026) — Closed-loop memory + action control; first open-domain revisited benchmark testing temporal stability across viewpoint changes. [Paper] [Code]
- WorldArena
- Omni-WorldBench
- Scaling Laws for Pre-training Agents and World Models (Nov 2024) — Establishes predictable scaling laws for agent pre-training; coefficients vary by implementation choices. [Paper]
- When Do Neural Networks Learn World Models? (Feb 2025) — Proves low-degree-bias models recover latent world variables through multi-task learning under mild assumptions. [Paper]
- General Agents Need World Models (Jun 2025) — Formally proves multi-step goal-directed agents mathematically require world models for generalization. [Paper]
- What Does it Mean for a Neural Network to Learn a 'World Model'? (Jul 2025) — Defines precise, testable criteria for verifying neural networks genuinely learn world models. [Paper]
- Compositional Generative Modeling: A Single Model is Not All You Need (Feb 2024) — Composing smaller specialized generative models outperforms monolithic models in efficiency and adaptability. [Paper]
- Video as the New Language for Real-World Decision Making (Feb 2024) — Proposes video as a unified interface for decision-making, analogous to language for LLMs. [Paper]
- Interactive Generative Video as Next-Generation Game Engine (Mar 2025) — Argues generative video models will replace traditional game engines for unlimited dynamic content. [Paper]
- Critiques of World Models (Jul 2025) — World models should simulate actionable possibilities for purposeful reasoning, not just prediction. [Paper]
- Beyond World Models: Rethinking Understanding in AI Models (Nov 2025) — Challenges the assumption that world model capabilities equate to human-like understanding. [Paper]
- Transformers Use Causal World Models in Maze-Solving Tasks (Dec 2024) — Shows transformers develop causally interpretable internal world models with asymmetric feature activation. [Paper]
- MetaOthello
- Towards Video World Models — In-depth blog on the trajectory from video generation to world simulation. [Blog]
- Jim Fan's World Model Thread — Influential X/Twitter thread defining world models in the context of modern AI. [Link]
Contributions are welcome! Please feel free to submit a Pull Request or open an Issue.
When adding a new entry, please follow the format below. Affiliation and extra links are optional. Sort chronologically within each section.
Research papers:
- **Model Name** (Affiliation, Mon Year) — **Key technique**; one-line differentiator. [[Paper](url)] [[Code](url)] [[Blog](url)] [[Website](url)]
Surveys & positions:
- **Paper Title** (Mon Year) — One-line summary. [[Paper](url)]
If you find this repository useful, please consider citing it:
@misc{awesomeworldmodel,
title={Awesome World Models for Video, Games \& 3D},
author={Xin (Kris) Gao and Contributors},
year={2026},
howpublished={\url{https://github.com/gxchris95/awesome-world-models-video-games-3d}},
}If you find this list helpful, please give it a star!