Innhold Show
PAN 2 is an open world model that lets an AI agent simulate what could happen before it acts. Give it a scene and an action, and it can generate up to 20 seconds of video showing a possible outcome. That is a much more interesting ambition than producing yet another pretty video clip.
The Institute of Foundation Models at MBZUAI has paired a language model that plans the events with a 29.2 billion-parameter diffusion transformer that turns the plan into motion and images. The weights, training code and inference code are available under Apache 2.0. That makes PAN 2 a rare release in a part of AI where “open” often means you can download something, but not enough to understand or extend the system.
I like this direction. A robot or agent that can imagine several outcomes before choosing an action is more compelling than one that simply hopes for the best. But PAN 2 is still research, not a finished safety engine. About two out of three multi-step plans missed at least one step in IFM’s own evaluation.
What is the PAN 2 world model?
PAN 2 is a general-purpose world model for physical AI. It accepts text and an optional starting image, then generates video at 704 x 1248 pixels and 24 frames per second. According to the official PAN 2 model card, one simulated sequence can last up to 20 seconds.
A conventional video model mainly tries to create a clip that looks plausible. A world model has a harder job: it is supposed to predict how a state changes when an action occurs. If a drawer opens, a ball drops or oil is poured into water, attractive images are not enough. The order, timing, objects and physical consequences all need to make sense.
That is why IFM describes PAN 2 as a simulator for “simulative reasoning.” An agent can propose several plans, simulate each one and select the outcome that appears best. On paper, that can help robots, AI agents, training systems and research into cause and effect. In practice, its value depends on whether the simulation is reliable enough to stop the agent from learning from a beautiful lie.
Two models divide the reasoning
PAN 2 separates conceptual reasoning from visual reasoning. First, a multimodal language model reads the observation and requested action. It writes a structured account of what should happen, who performs each action, when it occurs and how the camera moves. The large diffusion model then turns that description into video.
IFM’s technical overview calls the architecture Generative Latent Prediction. The language model works with the conceptual state, while the latent diffusion model fills in geometry, appearance and motion. A frozen Qwen3.5-9B provides the text encoding, and a Wan 2.2 VAE decodes the completed video latents into frames.
That may sound like an extra detour, but it is the point of the system. In IFM’s ablation tests, full-plan completion fell from 34% to 0% when conceptual reasoning was removed. Individual step completion dropped from 75.1% to 8.7%. The language model is not decoration in front of the video generator. It carries a large part of the plan understanding.
How open is PAN 2?
PAN 2 is unusually open. IFM has released the model weights, training and inference code, and the conceptual reasoner’s prompt and agent skill under Apache 2.0. The official PAN 2 repository includes configurations for pretraining and supervised fine-tuning, inference scripts and support for distributed training across multiple GPUs.
Researchers and developers can inspect the architecture, fine-tune the model and build their own variants without depending on a closed API. PAN 2 is also supported in Hugging Face Diffusers and vLLM. This is the kind of release that can genuinely move a field forward because other people receive more than a score table and a handful of carefully selected demonstrations.
Apache 2.0 applies to PAN 2’s own code and weights. Qwen3.5-9B, Wan 2.2 and the other third-party components retain their own licenses. Anyone considering commercial use should therefore inspect NOTICE and THIRD_PARTY_LICENSES as well. The openness is real, but it does not turn every component into one legally uniform package.
The results are promising, but this is not solved
IFM evaluated PAN 2 on 100 multi-step tasks. The model completed the entire plan in 34% of cases, compared with 32.5% for MiniMax-H3 in the same setup. It completed 75.1% of individual steps. That is promising for an open model, but 34% also means 66 out of 100 plans missed at least one step.
Across 300 single-action cases, PAN 2 received an overall score of 84.48%. MiniMax-H3 scored higher at 87.20% and also won the human preference evaluation. These figures come from IFM’s own evaluation setup. They tell us something useful about the model, but they are not an independent verdict.
The model card is refreshingly clear about the weaknesses. Long plans often fail, results are sensitive to prompt wording, and appearance can drift between clips during longer interactions. Text is also a coarse way to specify dexterous motion. “Open the drawer” is easy to describe. The exact contact point between every finger and an object requires far more precision.
The most serious limitation is that PAN 2 can skip physical intermediate states. The model card gives an example of pasta appearing in a colander before it has actually been drained into it. A clip like that may look reasonable to a person following the broad story, yet be useless to a robot that needs the state of the world to be correct at every moment.
Can PAN 2 run locally?
Yes, but “locally” means a powerful GPU workstation or server node here, not an ordinary laptop. The diffusion transformer contains 29.2 billion parameters, and the published transformer files occupy roughly 67 GB. The standard setup uses BF16 and loads the complete model into every inference worker, while block offloading is available as a slower fallback.
IFM does not publish one official VRAM requirement or a representative generation time in the model card, so it would be misleading to claim that a particular consumer GPU is guaranteed to handle it. The complete workflow also needs Qwen3.5-9B as its text encoder and the Wan 2.2 VAE. The standard configuration is firmly in data-center territory, even if quantization and offloading may eventually make experiments possible on smaller machines.
The published inference recipe uses 50 sampling steps, a flow shift of 9, guidance of 2.5 and BF16 precision. A default clip contains 241 frames, or about 10 seconds at 24 fps. The maximum is 481 frames, or 20 seconds. There is currently no official hosted inference provider on the Hugging Face model page, so you need to deploy it yourself or wait for someone else to offer access.
What can developers use PAN 2 for?
The clearest use is research into robots and agents that need to choose between actions. An agent can generate several candidate plans, ask PAN 2 to simulate them and use a separate evaluation mechanism to rank the outcomes. The model can also support work on physical and causal reasoning, synthetic training scenes, and experiments showing how small changes in timing alter an outcome.
Developers should start with evaluation, not autonomous control. PAN 2’s videos are predictions, not proof of what will happen. Give it tasks where a bad simulation can be detected and rejected. Do not give it sole responsibility for medical, legal or safety-critical decisions. IFM explicitly says that those uses require independent human review and validation.
It is equally important to evaluate the entire plan rather than checking whether the final frame looks correct. A simulation can arrive at a plausible ending while cheating along the way. In physical AI, the intermediate states are often the most important part. A robot arm cannot teleport pasta, tools or fingers between two convincing keyframes.
PAN 2 can also work as a text-to-video or image-to-video model, but it then competes with systems designed for visual quality, audio and fast production. I would not choose it merely because it can generate video. Its value is the explicit connection between a plan, time and a simulated physical outcome.
An open platform, not a finished answer
PAN 2 shows a direction I think will matter: AI agents that do more than write a plan in text and instead try to simulate the consequences before doing something in the world. IFM’s decision to publish both the weights and the training code under Apache 2.0 makes this release far more valuable than a closed demonstration.
At the same time, 34% full-plan completion is a research result, not a green light for autonomous action. The model is heavy, prompt-sensitive and capable of violating physical rules between the beginning and end of a sequence. That is no minor detail when the entire purpose is to give an agent a more reliable view of consequences.
Use PAN 2 as an open platform for experiments, comparisons and further development. Validate every simulation against real data, measure where and how plans fail, and never confuse a convincing video with certain knowledge. This is an exciting step for physical AI. It is simply not the last one.
Frequently asked questions
Is PAN 2 open source?
PAN 2’s own weights, training code and inference code are released under Apache 2.0. Third-party components including Qwen3.5-9B and Wan 2.2 retain their own licenses, so commercial users should also review the repository’s NOTICE and THIRD_PARTY_LICENSES files.
How long are PAN 2 videos?
PAN 2 can generate up to 20 seconds of video at 704 x 1248 pixels and 24 frames per second. It supports text-to-video and image-to-video generation, with an optional starting image providing the initial state for the simulated action.
Does PAN 2 need a separate language model?
Yes. The complete workflow uses a language model as the conceptual reasoner. It creates a time-grounded description of events, actors and camera motion before the diffusion model generates the video. IFM used Kimi K3 for its main evaluation, but other instruction-following models can fill the role.
Can PAN 2 control a robot directly?
PAN 2 should not be the sole decision system for direct or safety-critical robot control. Roughly two out of three evaluated multi-step plans missed at least one step, and the model can create visually plausible sequences that violate important physical intermediate states.
