Qwen-Drive-1.0-4B

[!Note] This repository contains the weights and configurations of Qwen-Drive-1.0 in the Hugging Face format. The accompanying code, demo data and documentation are released at QwenLM/Qwen-Drive-1.0.

Qwen-Drive-1.0 retains the architecture of the pretrained Qwen3.5 vision-language model and integrates 3D perception, visual question answering, and motion planning within a unified framework. The natively multimodal Qwen3.5-4B serves as the shared VLM, with two external modules attached: a BEV perception head jointly performing 3D object detection, semantic occupancy prediction and BEV map segmentation, and a Planning Expert that conditions on the shared VLM representations to generate future ego trajectories through flow matching. The unchanged VLM answers free-form questions about driving scenes. A staged training recipe combines driving supervision with general-purpose vision-language data, so the model acquires driving-specific competence while preserving broad visual understanding and instruction-following capability.

For more details, please refer to our Technical Report: Qwen-Drive-1.0.

Highlights

• Qwen-Drive-1.0 is the first vision-language foundation model for autonomous driving that unifies 3D perception and visual question answering at the pretraining stage and further extends to motion planning, while keeping the pretrained VLM architecture entirely untouched.
• An external BEV perception head serves as an explicit, inspectable 3D probe, jointly learning 3D detection, semantic occupancy prediction, and BEV map segmentation, equipping the same pretrained VLM with clear perception outputs while preserving highly competitive vision-language performance.
• A staged training and data recipe unifies cross-dataset labels, rewrites responses, filters samples for consistency, and combines driving data with general-purpose vision-language supervision, supporting domain adaptation while mitigating catastrophic forgetting.
• A Planning Expert tailored to pretrained VLM representations generates future ego trajectories with flow matching. Unified trajectory annotations enable joint training across multiple public driving datasets and yield highly competitive results across open-loop, pseudo-closed-loop, and closed-loop evaluations.
• Two Planning Experts are released: planner-sft supports both direct and reasoning planning; planner-rl is further reward-optimized on NAVSIM PDMS, WOD-E2E RFS and a displacement term, and runs best in the reasoning planning mode.

Motion planning

With planning samples assembled purely from public sources, Qwen-Drive-1.0 unifies the trajectory format across datasets and evaluates from open-loop prediction to closed-loop driving. The SFT model is already competitive across all benchmarks. After reinforcement learning, the model trades only a marginal open-loop displacement for comprehensive gains in human-preference alignment and closed-loop safety.

Vision-language understanding

Driving VQA. Qwen-Drive-1.0-SFT leads both general-purpose VLMs and driving or embodied specialists on driving question answering, with the sharpest spatial understanding and a more accurate sense of physical scale. The gain in causal reasoning is the most pronounced, and its driving-decision capability generalizes from broad driving data rather than memorizing specific scenarios.

General vision-language understanding. Large-scale driving training causes no evident catastrophic forgetting. Qwen-Drive-1.0-SFT largely preserves its general vision-language capability, performing on par with the base Qwen3.5-4B across the knowledge, reasoning, recognition and spatial understanding benchmarks, while well preserving its instruction-following capability.

3D perception

A single Qwen-Drive-1.0-SFT model produces coherent 3D detection, semantic occupancy, and BEV map segmentation that reflect genuine 3D structure rather than inheriting label noise. The BEV perception head is deliberately kept simple, so that it serves as an explicit, inspectable 3D probe of the shared VLM representations rather than a specialist aimed at advanced perception benchmarks.

Repository Contents

Everything ships in one directory. The VLM sits at its root, shared by every task, and each task head in a subfolder beside it.

Qwen-Drive-1.0-4B/          9.1 GB  the VLM, which on its own serves the VQA mode
├── planner-sft/            2.1 GB  Planning Expert, imitation-trained
├── planner-rl/             2.1 GB  Planning Expert after reward optimization
└── perception/             0.5 GB  BEV perception head

Quickstart

Install the inference code from the GitHub repository:

git clone https://github.com/QwenLM/Qwen-Drive-1.0 qwen-drive && cd qwen-drive
pip install -e . --no-build-isolation

Download the weights (the VLM at the root plus every task head in its subfolder):

hf download Qwen/Qwen-Drive-1.0-4B --local-dir Qwen-Drive-1.0-4B

Load the VLM with a Planning Expert attached and predict trajectories:

import torch
from qwen_drive import InferenceMode, QwenDriveForPlanning
from qwen_drive.benchmarks import read_scene_file
from qwen_drive.images import ImageArchive

model = QwenDriveForPlanning.from_pretrained(
    "Qwen-Drive-1.0-4B",
    planner="Qwen-Drive-1.0-4B/planner-rl",
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
).to("cuda").eval()

scene = next(
    read_scene_file(
        "data/demo/planning_scenes.jsonl",
        image_archive=ImageArchive.open("data/demo/frames.parquet"),
    )
).scene

result = model.run(InferenceMode.REASONING_PLANNING, scene=scene, num_samples=6)
print(result.reasoning)
print(result.trajectories.shape)   # (6, 50, 3) -> (x, y, heading), 5 s at 10 Hz

• VQA. Load the root directory without a planner and the VLM answers free-form driving questions on its own.
• Perception. Attach the BEV head with QwenDrivePerception.from_pretrained("Qwen-Drive-1.0-4B/perception"); it binds the same VLM and outputs 3D detections, occupancy and BEV map segmentation.
• planner-rl was reward-optimized only on reasoning-conditioned rollouts, so run it in the reasoning planning mode. planner-sft covers both direct and reasoning planning.

scripts/demo.py in the GitHub repository runs four bundled planning scenes and six perception frames end to end without any extra data.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{zhou2026qwendrive10initialstepvisionlanguage,
      title={Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving}, 
      author={Xin Zhou and Zongchuang Zhao and Zhibo Yang and Mingsheng Li and Humen Zhong and Shuai Bai and Du Chu and Ruizhe Chen and Zhaohai Li and Jun Tang and Qiuyue Wang and Mingkun Yang and Jiazhao Zhang and Dayiheng Liu and Dingkang Liang and Xiang Bai},
      year={2026},
      eprint={2609.00111},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.00111}, 
}