Q3.5-9B-GLM-5.1-v2.0
Q3.5-9B-GLM-5.1-v2.0 is a reasoning-capable 9B-parameter language model built on top of Qwen/Qwen3.5-9B. The model was trained through a multi-stage training pipeline using approximately 1.5K long-context GLM-5.1 math and science reasoning traces, along with additional high-quality reasoning traces, to improve long-form reasoning, mathematical problem solving, scientific analysis, and instruction-following capabilities.
[!NOTE] This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.
Key Highlights
- Qwen 3.5 Foundation: Built directly on top of Qwen/Qwen3.5-9B.
- Multi-Stage Training: Trained through multiple stages to progressively improve reasoning performance.
- Long-Context Reasoning: Incorporates approximately 1.5K GLM-5.1 long-context reasoning traces covering mathematics, science, and complex analytical tasks.
- General Reasoning Enhancement: Further trained on additional high-quality reasoning traces to strengthen instruction following and multi-step reasoning.
- Research-Focused Release: Designed for reasoning research, experimentation, and evaluation.
- Efficient 9B Deployment: Suitable for local inference and research environments.
Quick Start with Transformers
pip install transformers
pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model = AutoModelForCausalLM.from_pretrained(
"prithivMLmods/Q3.5-9B-GLM-5.1-v2.0",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"prithivMLmods/Q3.5-9B-GLM-5.1-v2.0"
)
messages = [
{
"role": "user",
"content": "Explain how a transformer model processes text."
}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=512
)
print(
tokenizer.decode(
outputs[0][inputs.shape[-1]:],
skip_special_tokens=True
)
)
Training Details
| Setting | Value |
|---|---|
| Base Model | Qwen/Qwen3.5-9B |
| Training Method | Multi-stage Supervised Fine-Tuning (SFT) |
| Maximum Sequence Length | 32,768 tokens (Long Context) |
| Training Precision | BF16 (Full Precision) |
| Training & Alignment Framework | TRL |
| Training Datasets | Jackrong/GLM-5.1-Reasoning-1M-Cleaned and additional high-quality reasoning datasets |
Model Files
| Resource | Link |
|---|---|
| Transformers Model | prithivMLmods/Q3.5-9B-GLM-5.1-v2.0 |
| GGUF (llama.cpp Quantizations) | https://huggingface.co/prithivMLmods/Q3.5-9B-GLM-5.1-v2.0-GGUF |
Intended Use
- Reasoning Research: Studying long-context reasoning and multi-stage training techniques.
- Mathematical Reasoning: Solving complex mathematical problems with multi-step reasoning.
- Scientific Reasoning: Performing structured scientific analysis and problem solving.
- Instruction Following: Evaluating and improving instruction-following capabilities.
- Local Deployment: Running efficient 9B reasoning models in research and experimentation environments.
Limitations
- Experimental Model: Behavior may differ from the base model in certain scenarios.
- Reasoning Artifacts: Complex reasoning chains may occasionally produce incorrect intermediate steps or conclusions.
- Training Biases: Performance reflects the characteristics and coverage of the reasoning datasets used during training.
Acknowledgements
-
Qwen/Qwen3.5-9B: Base model used for this project.
-
TRL - Transformers Reinforcement Learning: TRL is a full stack library providing tools to train transformer language models with methods including Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), Direct Preference Optimization (DPO), Reward Modeling, and more.
-
Transformers: Transformers provides state-of-the-art machine learning models for text, computer vision, audio, video, and multimodal tasks, supporting both inference and training.