Qwen3-VL-2B-Instruct-LLM
The language model of Qwen/Qwen3-VL-2B-Instruct,
extracted as a plain Qwen3ForCausalLM checkpoint (vision tower dropped).
The Qwen3-VL-2B text tower is architecturally identical to Qwen3-1.7B dense
(hidden 2048 / 28 layers / 16 Q + 8 KV heads / head_dim 128 / FFN 6144 /
vocab 151936 / tied embeddings). Its interleaved MRoPE assigns text tokens the
same index on all three axes, which reduces exactly to standard 1D RoPE, so
this export keeps the VL rope_theta (5e6) and drops rope_scaling.
Equivalence: text-only logits match the original VL model bit-for-bit in
fp32 (max |logit diff| = 0.0 on a held-out prompt).
Load with:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("shushengyang/Qwen3-VL-2B-Instruct-LLM")
Produced by personal/shusheng/mllm/misc/convert_qwen3vl_llm_to_qwen3.py
(amilabs monorepo); intended as an LLM initialization for ScaleWM MLLM
experiments (vision-language-pretrained LLM vs text-only Qwen3-1.7B).