Qwen2.5-VL-3B-Instruct with action tokens
This repository is a Qwen2.5-VL-3B-Instruct initializer whose tokenizer and
text vocabulary have been extended with 2,048 tokens named
<robot_action_0> through <robot_action_2047>. It is intended as a
starting point for StarVLA frameworks that discretize robot actions as tokens.
The packaged config.json identifies
Qwen2_5_VLForConditionalGeneration, a 2,048-dimensional, 36-layer text
backbone originating from Qwen2.5-VL-3B-Instruct.
Model and input contract
| Item | Value |
|---|---|
| Base model | Qwen/Qwen2.5-VL-3B-Instruct |
| Architecture | Qwen2_5_VLForConditionalGeneration |
| Added action tokens | 2,048 |
| Action-token IDs | 151665–153712 in the packaged tokenizer |
| Inputs | Same text, image, and video inputs as the base Qwen2.5-VL model |
| Output | Language-model logits over the extended vocabulary |
| Robot action head | None |
| Robot checkpoint | None |
The extension was produced with
starVLA/model/modules/vlm/tools/add_qwen_special_tokens.
Training and evaluation
No additional VLM or robot-policy fine-tuning is reported for this repository. The pre-existing Qwen2.5-VL parameters are retained apart from resizing the token embedding/output vocabulary for the new entries. The added action-token embeddings therefore should be treated as initialization, not as learned action semantics.
No robot benchmark result is associated with this repository.
Loading
Use the model and tokenizer from the same repository so their vocabulary sizes and token IDs remain aligned:
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
repo_id = "StarVLA/Qwen2.5-VL-3B-Instruct-Action"
processor = AutoProcessor.from_pretrained(repo_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
repo_id,
torch_dtype="auto",
device_map="auto",
)
Downstream training must define the action quantization/codebook that maps continuous robot actions to the 2,048 token indices.
Evidence and limitations
config.jsonverifies the Qwen2.5-VL architecture and 3B-class hidden dimensions;added_tokens.jsonverifies all 2,048 action-token entries.- This repository does not define normalization, action dimension, action horizon, robot embodiment, dataset, controller, or evaluation protocol.
- Token IDs are repository-specific. Mixing this tokenizer with the unextended base model or another action-token checkpoint can silently corrupt decoding.
- The model inherits the base model license, acceptable-use requirements, biases, and failure modes.