Uranus
In this repository, we present Uranus, a data-driven robot simulator built around a joint-trajectoryconditioned autoregressive diffusion model. Uranus offers three key capabilities:
- streaming, open-ended rollout, which receives future joint-position trajectories online and autoregressively generates one latent frame per step, corresponding to four RGB frames, without a fixed horizon
- low-latency generation, achieving 24 FPS after inference optimization
- scalable, extensible robot control, providing a unified interface for synchronized multi-view generation across diverse robot embodiments and camera configurations.
More technical details can be found in our technical report, and visualization demos are available on our blog.
Quickstart
Installation
Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
Create the environment and install dependencies
git clone https://github.com/D-Robotics-AI-Lab/Uranus-OSS.git
cd Uranus-OSS
uv sync
All commands in this repo are run through uv run python <cmd>, which resolves .venv/ automatically without manual activation.
Download model weights
| Models | Download Link | Notes |
|---|---|---|
| Uranus-1.3B | 🤗 Huggingface 🤖 ModelScope | SFT model, 384×640, 25 inference steps |
| Uranus-1.3B-Distillation | 🤗 Huggingface 🤖 ModelScope | Distilled model, 384×640, 4 inference steps |
You can download the weights with the Hugging Face CLI (pip install -U huggingface_hub if you don't have it):
# SFT model (17.5 GB) into ./weights/uranus-1.3b
hf download D-Robotics/Uranus-1.3B --local-dir ./weights/uranus-1.3b
# Distilled model (17.5 GB) into ./weights/uranus-1.3b-distillation
hf download D-Robotics/Uranus-1.3B-Distillation --local-dir ./weights/uranus-1.3b-distillation
For gated repos, set HF_TOKEN first (hf auth login or export HF_TOKEN=...). The same repos are mirrored on ModelScope — use modelscope download D-Robotics/Uranus-1.3B --local-dir ./weights/uranus-1.3b.
Uranus consumes converted weights from a single directory. Expected layout:
weights_dir/
├── dit.pt # SpatialTemporalWanModel state_dict
├── vae.pt # WanVideoVAE state_dict (loaded strict=False)
├── text_encoder.pt # WanTextEncoder (T5) state_dict
├── plucker_adapter.pt # Plücker-conditioning adapter
├── vace_patch_embedding.pt # skeleton controller patch embedding
├── tokenizer/ # HuggingFace tokenizer directory
└── metadata.json # model + inference metadata
Download test samples
We provide a set of ready-to-run test samples covering a variety of robot embodiments (ALOHA, ARX5, UR5, Franka, G1, DOS-W1, X5) and data sources (AgiBot World, DROID, RC-Table). Download and unpack them from Huggingface into examples/data/.
hf download D-Robotics/Uranus-Demo-Data --repo-type dataset --local-dir ./examples/data
Each episode lands in ./examples/data/<episode_id>/ and can be passed directly to main.py --sample-dir.
Each sample is a self-contained "XML-environment" directory:
sample_dir/
├── meta.json # prompt, cameras, mjcf_path, end_effectors, skeleton, fps
├── temporal.json # step_qpos: [{state, robot2world_transform}, ...]
├── mjcf/
│ └── <robot>.xml # robot MJCF with baked-in camera calibration & mounts
├── ref_images/
│ └── <camera>.png # one reference image per camera
└── gt/
└── <camera>.mp4 # ground-truth video per camera
Your own robot data can be converted into this same format and fed to inference as well, as long as it provides the per-frame qpos, the robot MJCF with calibrated cameras, and one reference image per camera.
Run inference
Run generation on a test sample with the pretrained model:
uv run python main.py \
--weights-dir <weights_dir> \
--sample-dir examples/data/000000 \
--output-dir ./output
Flag reference
| Flag | Required | Description |
|---|---|---|
--weights-dir | yes | Path to the converted weights directory (see Download model weights). Must contain metadata.json. |
--sample-dir | yes | Path to a single XML-environment sample (see Download test samples). |
--output-dir | no | Root directory for generated videos. Defaults to ./output. |
--num-chunks | no | Number of autoregressive chunks to roll out. Omit to automatically use every frame in temporal.json (trailing partial chunk is padded with the last frame, then trimmed back to the original length in the output). |
Hyperparameters are auto-resolved from metadata.json
num_inference_steps, step_length, default_height, default_width, and teacher_forcing_window_size are read from the checkpoint's metadata.json automatically — no extra flags are needed. Any flag passed explicitly on the CLI takes precedence over the metadata. For example, the SFT model defaults to 25 denoising steps, while the distilled model defaults to 4.
Outputs
Each run writes four aligned video streams per camera under --output-dir:
gen/— model-generated videogt/— ground-truth video (copied from the sample for comparison)skeleton/— rendered MuJoCo skeleton visualizationplucker/— Plücker-coordinate conditioning visualization (force RGB + ray-direction RGB)
Plus a preview.mp4 that stacks all four streams side by side for quick qualitative comparison.
License
This project is released under the Apache License 2.0. By using, distributing, or contributing to this repository, you agree to the terms and conditions of the license.
Acknowledgments
We would like to express our gratitude to the Wan and diffsynth Team for open-sourcing their code and models. Their contributions have been instrumental to the development of this project.
Citation
🌟 If you find our work helpful, please leave us a star and cite our paper.
@article{drobotics2026uranus,
author = {D-Robotics Large Model Team},
title = {Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI},
journal = {D-Robotics AI Blog},
year = {2026},
}
Contact Us
If you would like to leave a message to our research or product teams, feel free to join our WeChat groups!
