Qwen3-VL-2B — nuScenes Driving QA + Counterfactual (LoRA, merged)
LoRA adaptation of Qwen3-VL-2B-Instruct for front-camera driving visual question answering on nuScenes, combining NuScenes-QA (perception: existence / count / status / position) and OmniDrive reasoning + counterfactual QA ("if you follow trajectory X, what happens?"). Vision encoder frozen; LoRA on the LLM + aligner, then merged into a standalone checkpoint. Trained on driving data only (no general-domain replay).
Results (front camera, greedy decoding, vs stock Qwen3-VL-2B)
| Task | Stock 2B | This model | Δ |
|---|---|---|---|
| NuScenes-QA (exact-match, n=2000) | 27.55% | 52.10% | +24.5 |
| Counterfactual (reason-hit) | 50.3% | 58.7% | +8.4 |
General-ability retention: MMStar 43.2→44.8, AI2D 72.2→73.2, ScienceQA 74.4→80.0, ChartQA 80.4→76.8, RealWorldQA 68.2→63.2 (largely preserved; several improve).
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
m = AutoModelForImageTextToText.from_pretrained("HqH1111/Qwen3-VL-2B-NuScenes-DrivingQA", torch_dtype="auto", device_map="auto")
p = AutoProcessor.from_pretrained("HqH1111/Qwen3-VL-2B-NuScenes-DrivingQA")
Training: Qwen3-VL-2B-Instruct · ViT frozen · LoRA r64/α128 (all-linear) · bf16 · 3×A100 · ms-swift · 50k samples (NuScenes-QA 28k + OmniDrive 22k) · 2 epochs. Non-commercial research use (nuScenes / OmniDrive terms).