Qwen3.5-2B-OnlyText
Text-only causal language model derived from Qwen/Qwen3.5-2B by removing the vision and audio stacks (their towers/embedders and projector weights) and the multimodal special tokens. The text backbone and LM head are preserved, and the MTP draft head is preserved.
Details
- Base model: Qwen/Qwen3.5-2B
- Architecture:
Qwen3_5ForCausalLM - Parameters: 1.94B
- Layers: 24 · Hidden size: 2048
- MTP head: preserved
- Weights:
bfloat16
Attribution
This model is a derivative of Qwen/Qwen3.5-2B by the Qwen team, released under the apache-2.0 license. All credit for the underlying weights and capabilities belongs to the original authors; this repository only removes modalities, it does not add new training.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("OnlyTextLLMs/Qwen3.5-2B-OnlyText")
tokenizer = AutoTokenizer.from_pretrained("OnlyTextLLMs/Qwen3.5-2B-OnlyText")