Qwen3.8-27B-OnlyText
Text-only causal language model derived from Qwen/Qwen3.8-27B by removing the vision and audio stacks (their towers/embedders and projector weights) and the multimodal special tokens. The text backbone and LM head are preserved, and the MTP draft head is preserved.
Details
- Base model: Qwen/Qwen3.8-27B
- Architecture:
Qwen3_5ForCausalLM - Parameters: 27.32B
- Layers: 64 · Hidden size: 5120
- MTP head: preserved
- Weights:
bfloat16
Attribution
This model is a derivative of Qwen/Qwen3.8-27B by the Qwen team, released under the apache-2.0 license. All credit for the underlying weights and capabilities belongs to the original authors; this repository only removes modalities, it does not add new training.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("OnlyTextLLMs/Qwen3.8-27B-OnlyText")
tokenizer = AutoTokenizer.from_pretrained("OnlyTextLLMs/Qwen3.8-27B-OnlyText")