AgentGarten renderer
The real-time neural renderer of AgentGarten: Code Worlds for Evolving Agents. A code world (a simulator or a game engine) keeps the state and runs the rules; this model turns the depth and surface normals it exports into the observations an agent sees, block by block.
- Paper: https://huggingface.co/papers/2610.12374
- Blog: https://mirros.ai/blog/worlds-for-evolving-agents
- Technical report: https://mirros.ai/report/agent-garten.pdf
- Project page: https://mirros-lab.github.io/agent-garten
- Code: https://github.com/MirroS-Lab/AgentGarten
What is in this repository
| File | Contents |
|---|---|
student.safetensors | The few-step autoregressive generation tower (7.0 B parameters, BF16), distilled with Adversarial Forcing |
und.safetensors | The frozen text tower of Cosmos3-Nano, unchanged |
text_tokenizer/ | The Cosmos3-Nano text tokenizer, unchanged |
config.yaml, manifest.json | Network configuration and the file list the loader reads |
The model starts from Cosmos3-Nano. It was adapted to depth and normal conditions, trained as a block-causal autoregressive model, and distilled into a four-step student. Video is encoded and decoded with the Wan 2.2 VAE, which is not included here.
Use
Install the AgentGarten code, then:
hf download MirroS-Lab/AgentGarten-renderer --local-dir weights/AgentGarten-renderer
hf download Wan-AI/Wan2.2-TI2V-5B Wan2.2_VAE.pth --local-dir weights/Wan2.2-TI2V-5B
from wm.inference.cosmos3 import inspect_artifact, load_artifact
from wm.inference.serving import prepare_serving
from wm.networks.cosmos3.streaming import Cosmos3Stream, StreamPolicy
artifact = load_artifact(inspect_artifact("weights/AgentGarten-renderer"))
prepare_serving(artifact.student)
stream = Cosmos3Stream(artifact.student, StreamPolicy())
The repository's README shows the streaming loop: each block of four latent frames is denoised in four steps and then committed to the key-value cache, so the conditions of the next block can depend on what was just generated.
License
The weights derive from Cosmos3-Nano and are released under the same OpenMDW License Agreement, version 1.1. The text tower and the tokenizer are NVIDIA's files, redistributed unchanged under that license.