RLDX-1

Paper  ·  Project page  ·  Code  ·  Models

RLDX-1 is a general-purpose Robot Foundation Model designed for dexterous manipulation. Powered by a Multi-Stream Action Transformer (MSAT), it seamlessly unifies multimodal perception (visual + tactile), high-DoF actuation, and memory-aware decision-making in a single architecture. RLDX-1 achieves state-of-the-art performance across diverse simulation benchmarks and is fully validated on real-world hardware.

This repository hosts RLDX-1-PT-IMG: a lightweight, image-input variant of the RLDX-1-PT, which uses 4-frame video inputs. This trades a minimal drop in success rate for a substantially lighter and faster policy, making it well suited to real-time and resource-constrained deployment. It is pre-trained on the same broad mixture of public manipulation corpora, providing a lightweight starting point for rapid experimentation on new embodiments and tasks.

Highlights

• Multi-Stream Action Transformer (MSAT). Cognition, physics, and action each get a dedicated stream coupled by joint self-attention — an extension of MM-DiT to action modeling.
• Motion awareness. Multi-frame observations + a motion module capture temporal dynamics; intermediate VLM layers compress video tokens to keep the policy efficient.
• Long-term memory. A memory module fuses past cognition features with the current ones for history-grounded decisions beyond a short multi-frame window.
• Physical sensing. Tactile and torque enter as a dedicated physics stream; the decoder is jointly trained to predict future physical signals.
• Three-stage training. Pre-training (generalization) → mid-training (functionality) → post-training (task adaptation), with synthetic data augmenting rare manipulation scenarios.
• Real-time inference. Static graph capture + custom fused kernels bring the all-modality model to 43.7 ms / step on RTX 5090 (1.63× speedup, >22 Hz).

Released Checkpoints

This card describes RLDX-1-PT-IMG (vision foundation checkpoint). The full RLDX-1 model family:

Quick start

git clone https://github.com/RLWRLD/RLDX-1.git
cd RLDX
uv sync --python 3.10
uv pip install -e .

Inference (single step)

from rldx.policy.rldx_policy import RLDXPolicy
from rldx.data.embodiment_tags import EmbodimentTag

policy = RLDXPolicy(
    model_path="RLWRLD/RLDX-1-PT-IMG",
    embodiment_tag=EmbodimentTag.OXE_FRACTAL,
    device="cuda:0",
)

action = policy.get_action(observation)

RLDX-1-PT-IMG is pretrained on a multi-source mixture, so for direct inference pair it with the embodiment tag matching your data source — e.g. OXE_FRACTAL, OXE_BRIDGE_ORIG, OXE_DROID, GALAXEA, AGIBOT_GRIPPER, AGIBOT_DEXHAND, NEURAL_GR1, HUMANOID_EVERYDAY_G1, HUMANOID_EVERYDAY_H1, etc. For custom robots, finetune.

Finetune from RLDX-1-PT-IMG

uv run python rldx/experiment/launch_train.py \
    --base-model-path RLWRLD/RLDX-1-PT-IMG \
    --dataset-path /path/to/your/dataset \
    --embodiment-tag GENERAL_EMBODIMENT \
    --video-length 4 --n-cog-tokens 64 \
    --global-batch-size 64 --learning-rate 1e-4 \
    --max-steps 60000 --save-steps 5000 \
    --output-dir ./outputs/my_finetune

To enable add-ons (memory / motion / physics) see the recipes in the main README and the training.md guide.

Model details

• Architecture: Multi-Stream Action Transformer (MSAT) policy with a Qwen3-VL vision-language backbone, cognition-token perceptual summary, optional Transformer memory, motion module, and tactile/torque physics encoder/decoder. Trained with flow matching.
• Inputs: A single RGB image per camera view (video_length=1, vs the 4-frame video input of RLDX-1-PT), state proprioception, language instruction.
• Outputs: Action chunks of length 16 (action_horizon=16).
• Backbone: Qwen/Qwen3-VL-8B-Instruct.
• Add-on modules: memory / motion / physics are dormant in this checkpoint (use_memory=use_motion=use_physics=false) and only activate when the corresponding flags are wired during finetuning (see RLDX-1-MT-ALLEX).
• Pretraining data: A mixture of public manipulation corpora, covering Open X-Embodiment (OXE) datasets (DROID, Bridge, Fractal, Language Table, …) plus Galaxea, AgiBot World (Gripper + Dexhand), ActionNet, Neural-Curated GR-1 humanoid trajectories, and Unitree G1 / H1 from HumanoidEveryday.

For a full architectural walkthrough see docs/architecture.md.

Intended use & limitations

Intended use. Research on robotic manipulation, finetuning on custom embodiments, simulation benchmarking, and non-commercial real-robot deployment under the conditions of the RLWRLD Model License v1.0.

Out of scope. Commercial deployment, military or weapons applications, non-consensual surveillance, and any use that violates applicable laws or regulations. See LICENSE.md §3.5 for the full list.

Limitations. Performance depends heavily on embodiment match and data distribution. The pretrained checkpoint is OXE-conditioned and is not guaranteed to work zero-shot on novel embodiments without finetuning. Memory, motion, and physics modules are dormant in RLDX-1-PT-IMG and only activate when the corresponding flags are wired during finetuning (see RLDX-1-MT-ALLEX).

Citation

@article{rldx2026,
  title={RLDX-1 Technical Report},
  author={Kim, Dongyoung and Jang, Huiwon and Koo, Myungkyu and Jang, Suhyeok and Kim, Taeyoung and others},
  year={2026},
  note={RLWRLD},
  eprint={2605.03269},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2605.03269}
}

License

Released under the RLWRLD Model License v1.0 — a non-commercial license with attribution and share-alike requirements. See LICENSE.md for the full text. By using this model you agree to those terms, including the use restrictions in §3.5.