.jpg?width=900)
Illustrative image: a robotics laboratory. PAN 2's official visuals are demo videos on the Institute of Foundation Models' website. Photo: Anders Sandberg, CC BY 2.0, via Wikimedia Commons.
TL;DR: PAN 2 open world model explained: on 10 November 2026, MBZUAI's Institute of Foundation Models released PAN 2, an open 29-billion-parameter world model that simulates what happens after an action — as video, grounded in physics, time and space. It leads six tested models on multi-step plans, but two in three plans still fail somewhere, and it needs data-centre GPUs.
Key Highlights
- The Institute of Foundation Models (IFM), established by Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), released PAN 2 on 10 November 2026 — an open, general-purpose world model for physical intelligence.
- Give it a starting image and an action described in words, and PAN 2 simulates up to 20 seconds of 704×1248 video at 24 frames per second showing what happens next.
- It splits the job in two: a language model plans the next world state as a structured, time-stamped caption, and a 29-billion-parameter diffusion transformer renders that plan into video.
- On IFM's own tests, PAN 2 completed 34.0% of full multi-step plans — the best of six models tested — and ranked first among open-weight models on the Physics-IQ Verified leaderboard.
- Weights, training code and inference code are all released under the Apache 2.0 licence. The catch: the model is 86 GB of weights and needs data-centre GPUs, not a laptop.
What happened?
On 10 November 2026, the Institute of Foundation Models (IFM) — the global AI research lab established by Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) — released PAN 2, a foundation world model for physical intelligence. The announcement was datelined Sunnyvale, California, where IFM runs one of its three labs; the others are in Abu Dhabi and Paris.
The idea is simple to state and hard to build. A language model predicts the next word. A world model predicts the next world: show it a scene, describe an action in text, and it simulates a video of what happens next — grounded, in IFM's words, in time, physics and space. Swap a strawberry for a rubber duck in the starting scene, or change the liquid being poured, and the splash changes with it, because the model is predicting a physical consequence rather than retrieving a clip.
IFM calls this simulative reasoning: an agent imagines the outcomes of its candidate actions before choosing among them, so it can learn and plan without costly, slow or irreversible real-world trials. As IFM founder Eric Xing put it in the announcement: "A language model learns the world as it has been written down. That is book knowledge, and it takes you a long way, but you cannot learn to swim from a book."
PAN 2 succeeds PAN, IFM's first world model, released in November 2025. It is built on the Generative Latent Prediction (GLP) architecture set out in IFM's Critique of World Models paper, which splits prediction into a reasoning half and a rendering half — and that split is the whole design.
PAN 2 open world model explained: two kinds of reasoning
PAN 2 is not one model doing one thing. It is a pipeline of two backbones that pass a written description of the world between them.
Conceptual reasoning. A large language model reads the starting observation and the requested action, then writes the next world state as a structured, time-grounded caption: the events in order, each action bound to the actor performing it, and the camera motion. Any instruction-following language model can fill this role; IFM's technical report uses Kimi K3 by default, and its evaluations also used Claude Opus 5.5 and GPT-6 Astra as the reasoner.
Perceptual reasoning. A 29-billion-parameter multi-modal diffusion transformer (MMDiT), trained from scratch by IFM, takes that caption — mapped into conditioning features by a frozen Qwen3.5-9B text encoder — plus the starting frame, and predicts the video latents the caption leaves open: contact, deformation, motion, how a wave breaks or a hand grips a pipette. A Wan 2.2 VAE decoder renders those latents into frames.
The ablation study shows the language model is doing real work, not just tidying the prompt. With no conceptual reasoning at all — the action sent straight to the diffusion model — single-action scores fell by nearly half and full-plan completion dropped to zero. A format-only rewrite, where the language model restructured the input without reasoning about how the world evolves, recovered most of the single-action gap but only part of the multi-step gap.
| Spec | PAN 2 |
|---|---|
| Model | 29-billion-parameter dense multi-modal diffusion transformer (MMDiT), trained from scratch |
| Inputs | Text action, plus an optional starting image |
| Output | Up to 20 seconds of 704×1248 video at 24 fps (241 frames, about 10 seconds, by default) |
| Reasoning backbone | Any instruction-following LLM (Kimi K3 by default); frozen Qwen3.5-9B text encoder; Wan 2.2 VAE decoder |
| Training data | Hundreds of millions of annotated real-world videos; gameplay, animation and simulation footage excluded after the earliest training phase |
| Licence | Apache 2.0 — weights, full training code and inference code |
| Runs on | Data-centre GPUs; each worker holds the full 29B model in BF16 (86.17 GB of weights) |
Training the diffusion backbone from scratch at this scale needed a new optimiser. PAN 2 uses a hybrid built around per-head Muon: attention query, key and value weights are split into head-sized blocks and updated independently, while attention-output and feed-forward weights use matrix-wise Muon and the rest use AdamW. In scaling ablations from 185 million to 1.52 billion parameters, per-head Muon reached the same validation loss with 65–71% less compute than AdamW alone. It is the same efficiency question that deletion tuning, which matches full fine-tuning by deleting a fraction of a percent of weights, attacks from the opposite direction.
Illustrative image: a robot arm at work in an autonomous laboratory — the kind of setting PAN 2 is built to simulate before a real robot acts. Photo: National Institute of Allergy and Infectious Diseases, public domain, via Wikimedia Commons.
How PAN 2 scored — and where it still falls short
IFM judges PAN 2 mainly by automated checks of what its simulations show, not by how good the video looks. Its own test suite covers 300 single-action cases and 100 multi-step tasks across six dimensions of physical intelligence: physical consistency, causal coherence, entity persistence, constraint satisfaction, viewpoint invariance and objective dynamism. That is supplemented by the public Physics-IQ and RBench benchmarks, plus blinded human comparisons. All of the numbers below are self-reported by IFM, from its own test suite and the model card.
| Benchmark (IFM-reported) | PAN 2 | Closest rival |
|---|---|---|
| Full-plan completion (100 multi-step tasks) | 34.0% — highest of six models | MiniMax-H3, 32.5% |
| Step completion | 75.1% | MiniMax-H3, 75.1% (level) |
| Six-dimension aggregate | 84.5% | MiniMax-H3, 87.2% — leads in all six categories |
| Physics-IQ Verified (public leaderboard) | 49.38 — first among open-weight models, third overall | Closed model FLUX 3, 51.11 |
| RBench (robotics) | 57.04 | MiniMax-H3, 56.16 |
| Human preference (Elo) | 1,595 | MiniMax-H3, 1,620 |
The honest reading is in the gap between the two headline numbers. PAN 2 completes 75.1% of individual steps but only 34.0% of full plans — about two in three multi-step plans contain at least one step that is not carried out. Errors compound across a plan, and IFM's own fitted curve shows the field sitting where that curve turns steep: raising step completion from 75% to 90% would roughly double full-plan completion. MiniMax-H3 still wins single actions, the six-dimension aggregate and human preference. And long-term world consistency — a world that grows and stays coherent as an agent explores it — remains an open challenge, which IFM says is the subject of an upcoming post.
Two training decisions are worth knowing about. First, IFM excluded synthetic content — gameplay, animation and rendered simulation — after the earliest training phase, so the model learns dynamics from the real world; a 9-billion-parameter test model trained without it won 58% of blinded pairwise votes. Second, reward-based learning after the main training run lifted causal coherence by 8.5 points and full-plan completion by 3.7 points, but cost 3.0 points of viewpoint invariance and 2.5 points of physical consistency — so the final model is a weighted average of the checkpoints before and after reward training, to keep the gains while limiting the damage.
There is also a safety footnote. IFM strongly recommends its separate pan2-guardrail: in its tests, the guardrail cut unsafe outputs from harmful text-to-video requests from 33.9% to 1.5% — but it also blocked 11.5% of benign requests. Open weights mean the guardrail is a recommendation, not an enforcement.
Why it matters for UAE and India readers
The UAE angle is direct: PAN 2 comes from a lab established by MBZUAI, the Abu Dhabi research university, with IFM's labs spanning Abu Dhabi, Silicon Valley and Paris. It is the most significant open release yet from the UAE's AI research ecosystem, and it is fully open — weights, training code and inference code under Apache 2.0 — at a time when most frontier video and world models stay behind APIs.
For India, the relevance is practical rather than ceremonial. Indian robotics startups, warehouse-automation teams and university labs can download and adapt PAN 2 without a licence fee or an API key, and use it as a rehearsal space: test a robot's plan in simulation before the real arm moves, or train an agent on simulated episodes before deploying it. The same simulative-reasoning idea is already reshaping adjacent fields — DeepMind's AlphaProof Code uses teams of AI agents to formally verify software, and WikiTrace gives LLM agents a memory that links lessons to their source runs — and PAN 2 supplies the missing half: a world for those agents to practise in.
The barrier is hardware, and it is steep. Each worker holds the full 29-billion-parameter model in BF16, and independent coverage of the vLLM serving recipe notes that thirteen seconds of video takes roughly 119 seconds on four B300 GPUs. That puts hands-on use in the hands of labs and companies with data-centre access, not individual developers — though the Apache 2.0 licence means cloud providers and Indian research institutions can host it for others.
Should you care?
✅ Yes, if you build robots, agents or embodied AI. PAN 2 is the most open world model available: full weights and training code, the best multi-step planning score of the models IFM tested, and a clear, honest account of its limits. If your work involves a machine acting in the physical world, this is a tool worth evaluating.
⏳ Wait, if you want a video generator. This is not a casual text-to-video toy. It needs data-centre GPUs, structured prompts (free-form wishes need rewriting through IFM's prompt-rewriter skill), and even then two in three long plans fail somewhere. Watch the space — but do not plan a product around today's 34%.
How to try PAN 2
PAN 2 is available now through IFM's website. The weights, training code and inference code are all under Apache 2.0, and you can run the model through Hugging Face Diffusers, vLLM, FastVideo or the repository's own generate.py script. A clip defaults to 241 frames — about ten seconds — with a 50-step sampler, flow shift 9 and guidance 2.5. Prompts work best as structured captions: IFM ships a Conceptual Reasoner skill for Claude Code and Codex that rewrites free text into the format the model expects, with a hard caption budget of 1,536 tokens. There is no price and no waitlist; the cost is the hardware. IFM's technical blog also publishes worked examples — a paper plane folded one step at a time, and an agent that learns, through six simulated attempts, that one cap of a tube is glued shut while the other comes off — which are the clearest demonstrations of what simulative reasoning actually looks like.
FAQs
What is PAN 2?
PAN 2 is an open, general-purpose world model from MBZUAI's Institute of Foundation Models, released on 10 November 2026. Given a starting image and an action described in text, it simulates up to 20 seconds of video showing what happens next, grounded in physics, time and space. Its 29-billion-parameter diffusion model is released with full training and inference code under Apache 2.0.
What is a world model?
A world model is an AI system that simulates how the world changes in response to actions. A language model predicts the next word; a world model predicts the next world. That lets AI agents rehearse plans in simulation — asking "what happens if I do this?" — before acting in the real world, where trials can be costly, slow or irreversible.
Is PAN 2 open source?
Yes. IFM released PAN 2 with its video model weights, full training code, inference code, a technical blog and documentation, all under the Apache 2.0 licence. Few world models are released with their full training code, which is what makes this release notable beyond its benchmark scores.
Can I run PAN 2 on my own computer?
Realistically, no — not on a laptop or a typical desktop. The model is 29 billion parameters, its weights total 86.17 GB, and each worker holds the full model in BF16 on data-centre GPUs. You can run it through Hugging Face Diffusers, vLLM or FastVideo if you have that hardware, or use a hosted deployment from a provider that does.
Takeaway
PAN 2 is not a better video generator. It is a rehearsal space for machines: a language model that plans what should happen, a diffusion model that renders it, and an open licence that lets anyone inspect both. At 34% full-plan completion it is plainly not solved — but it is the first open world model whose makers published exactly how far it gets, where it fails, and the code to improve it.
If you work in robotics or agent research, the most useful next step is IFM's technical blog itself — the worked examples of an agent learning through simulation are the clearest picture yet of what this kind of model is for.
Sources
- Institute of Foundation Models — PAN 2 Press Release (10 November 2026)
- Institute of Foundation Models — PAN 2 Technical Blog (10 November 2026)
- MarkTechPost — MBZUAI's IFM Releases PAN 2, an Open 29B World Model for Physical AI (10 November 2026)
- Xing, Deng & Hou — Critique of World Models (arXiv:2507.05169)
Comments
Post a Comment