AlayaWorld

AlayaWorld is a video world model that turns a single starting image into a camera-controllable, minute-long environment you can steer in real time. It comes from Alaya Lab, a Tokyo-based research group under Shanda AI Research (SARI), the AI institute founded by Chinese entrepreneur Chen Tianqiao. The team released it fully open, weights included, on July 7, 2026.

Overview

Video generation models are good at making a clip look right for a few seconds. Keeping a scene coherent while a user steers a camera, casts a spell, or walks back to a spot they already visited for a full minute is a different problem, and it's the one AlayaWorld's authors set out to solve. The project shipped in stages: a short arXiv paper on July 7, 2026 that hit #1 on Hugging Face's Daily Papers the next day, inference code and pretrained weights on July 16, and a full technical report on July 20-21.

The system takes an image, a camera path, and a text prompt, then rolls out video chunk by chunk - about 1.33 seconds per chunk at 24fps - along that path. Every few chunks a new prompt can be injected mid-generation ("fireworks burst over the rooftops"), and the underlying scene keeps its geometry. That combination of steerability and persistence is what separates a world model from a text-to-video generator, and it's the same bet NVIDIA is making with Cosmos 3 and NVIDIA NVLabs made with the far smaller SANA-WM.

Alaya Lab isn't a one-off shop. The same team published WildWorld, an action-conditioned dataset built from an AAA action RPG, and separately runs iWorld-Bench, the evaluation suite AlayaWorld reports its own numbers against. That's worth flagging up front: a lab publishing both the model and the benchmark it wins on is not disqualifying, but it's a reason to read the comparison table with a critical eye rather than take it at face value.

AlayaWorld's project page teaser: the same underlying model rendering pyramid, coastal, forest, and urban scenes across seven visual styles, with real-time camera control and prompt-triggered events. Source: alaya-lab.github.io

Key Specifications

The dependency list is longer than the parameter count suggests. Alongside its own weights, AlayaWorld needs Google's Gemma-3-12B as a text encoder and ByteDance's Depth-Anything-3 for its spatial-memory branch - neither ships in the repo, both have to be pulled separately. It's not a single-download model.

Benchmark Performance

AlayaWorld reports its results on iWorld-Bench, a benchmark from the same lab covering 14 interactive world models across Generation Quality, Trajectory Following, and Memory Ability. The full technical report puts AlayaWorld against Cosmos, HunyuanVideo-1.5, YUME 1.5, Matrix-Game 2.0, and HY-World 1.5.

HY-World 1.5 is the closest competitor by a wide margin - the gap on motion smoothness is basically noise, and memory symmetry is close too. Where AlayaWorld pulls ahead clearly is color and brightness stability across a rollout, which lines up with the paper's central claim: its error bank training specifically targets the drift that makes long generations shift hue or dim over time. Matrix-Game 2.0 lags on almost every axis here, and Cosmos, despite being the best-known name in the comparison, trails on memory symmetry and trajectory accuracy - a reminder that brand recognition and long-horizon world modeling are different skills.

One caveat worth repeating: this evaluation was run at 480p to match the benchmark's input frames, not at the 720p AlayaWorld is capable of, and the benchmark itself comes from Alaya Lab. Independent third-party replication on iWorld-Bench, or a comparable evaluation from someone outside the lab, hasn't happened yet as of this writing.

Key Capabilities

The interaction model has two channels. A rendered 3D cache with AdaLN camera modulation handles trajectory-aware navigation - move the camera, the geometry that was already created stays put and reprojects correctly. Chunk-level prompt switching handles events - a new text prompt can fire mid-rollout without restarting the scene, which is how the project demo triggers spells and weather changes over a static pyramid environment.

Long-horizon stability doesn't come from a bigger context window; there isn't one, in the language-model sense. It comes from training on deliberately corrupted history - feeding the model its own drifted outputs during training so it learns to correct rather than compound errors - combined with an "error bank" that reinjects accumulated artifacts back into training targets. Combined with the persistent sink frame and compressed temporal history, that's what gets rollouts to roughly a minute without the visible drift that shows up in most autoregressive video generation past 10-20 seconds.

Runtime efficiency is the other half of the pitch. AlayaWorld's distillation - blending distribution-matching distillation, self-forcing++, and consistency distillation - cuts the teacher's ~30-step sampling process down to 4 steps per chunk. That's the difference between "generates a usable minute-long clip" and "generates a minute-long clip you could plausibly interact with in something close to real time," though the repo doesn't publish end-to-end latency numbers or specific GPU throughput figures for the released checkpoint.

Pricing and Availability

There's no pricing tier, API, or hosted endpoint. AlayaWorld ships as open weights on Hugging Face (merged_infer.safetensors) with inference code on GitHub, and running it requires a CUDA GPU with PyTorch 2.6+ for flex_attention support, plus the separately-downloaded Gemma-3-12B text encoder and Depth-Anything-3 weights.

The license is the detail to get right before building anything on top of this. Some early aggregator coverage described the release as Apache 2.0 full-stack open source. The repository's own LICENSE and NOTICE files say otherwise: because the weights are fine-tuned from LTX-2.3, the entire project - code and weights - ships under the LTX-2 Community License Agreement, restricted to academic research and non-commercial use. Organizations with $10 million or more in annual revenue need a separate commercial license from Lightricks for any LTX-2 derivative, AlayaWorld included. Training code and training data are listed on the project roadmap as not yet released.

Strengths

• Leads or ties the field on most iWorld-Bench dimensions among open-weight interactive world models, particularly color and brightness stability over long rollouts
• Full inference code and weights available within days of the first paper, unusually fast for a research release
• Explicit engineering for the failure mode (drift/collapse past 10-20 seconds) that limits most competing systems
• Distillation drops sampling from ~30 steps to 4 per chunk without a published quality collapse on the reported metrics

Weaknesses

• License is academic/non-commercial only, not the permissive open-source terms some coverage implied - a real constraint for any product use
• Benchmark is self-published by the same lab; no independent third-party replication yet
• Requires three external weight downloads (Gemma-3-12B, Depth-Anything-3, plus its own checkpoint) - not a single-file deployment
• Training code and training data remain unreleased as of this writing, limiting reproducibility
• No published end-to-end latency or GPU throughput numbers for real-time interactive use

Related Coverage

• LTX-2.3 - the base video diffusion model AlayaWorld is fine-tuned from
• NVIDIA Cosmos 3 - the physical-AI world model AlayaWorld benchmarks against and loses to on some metrics, beats on others
• NVIDIA SANA-WM - a smaller, single-GPU world model built for robotics rather than interactive play
• Video Generation Benchmarks Leaderboard - where open-weight video models rank overall
• Odyssey Raises $310M as Amazon Bets on World Models - context on the wider world-model funding push
• LeCun Raises $1B Seed to Build AI Beyond LLMs - the broader industry bet that world models matter as much as language models

FAQ

What is AlayaWorld?

An open-weight video world model from Alaya Lab (Shanda AI Research) that turns an image, camera path, and text prompt into an interactive, minute-long video environment you can steer and edit mid-generation.

Can I use AlayaWorld commercially?

Not under the current license. It ships under the LTX-2 Community License Agreement, restricted to academic research and non-commercial use. Commercial use requires licensing LTX-2 directly from Lightricks.

How is AlayaWorld different from a text-to-video model?

Text-to-video models produce a fixed clip from a prompt. AlayaWorld generates chunk by chunk in response to live camera movement and mid-rollout prompt changes, while keeping the scene's geometry consistent as you revisit places.

What hardware does AlayaWorld need?

A CUDA GPU with PyTorch 2.6 or later for flex_attention support, plus separate downloads of Google's Gemma-3-12B text encoder and ByteDance's Depth-Anything-3 depth model. No official minimum VRAM figure is published.

How long can an AlayaWorld session run before quality degrades?

The paper reports roughly 60 seconds of continuous, spatially coherent rollout (about 45 chunks at 1.33 seconds each), longer than the other open-weight interactive world models it compares against on iWorld-Bench.

Sources

• AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (arXiv 2607.18367)
• AlayaWorld: Long-Horizon and Playable Video World Generation (arXiv 2607.06291)
• AlayaWorld GitHub Repository - AlayaLab
• AlayaWorld Model Weights - Hugging Face
• AlayaWorld Paper Page - Hugging Face Daily Papers
• AlayaWorld Interactive Project Page and Demo
• Open-Source World Model AlayaWorld Sustains Interactive Play Past the One-Minute Mark - Tech Times
• WildWorld Dataset - AlayaLab
• Shanda Scientific Research