# QwennAI / Qwen3.9-245B-A29B 🚀✨
## Introduction 🌟🔥
**Qwen3.9-245B-A29B** is an advanced open-weights Mixture-of-Experts (MoE) foundation model from the Qwen series. Built with hybrid thinking mechanisms and sparse routing, it features **245B total parameters** while activating only **29B parameters** per forward pass. This architecture enables frontier-grade reasoning, long-horizon tool execution, and code synthesis at high throughput.
### Highlights 💡
* **Sparse MoE Architecture:** 245B total parameters with 29B active per token across routed expert layers.
* **Hybrid Reasoning Control:** Native support for dual-mode generation (`thinking` with chain-of-thought vs. efficient `instruct` mode).
* **Ultra Long-Context Support:** 131,072 native context window, extendable up to 1,000,000 tokens with YaRN / Dual-Chunk Attention.
* **Agentic & Tool Calling:** Enhanced AST parsing for JSON-based function calling, multi-turn tool loops, and code interpreters.
---
## Model Overview 🛠️📐
* **Architecture:** Sparse Mixture-of-Experts (MoE) Transformer
* **Total Parameters:** 245B
* **Activated Parameters:** 29B
* **Number of Layers:** 96
* **Hidden Dimension:** 8,192
* **Attention Heads:** 64 for Q, 8 for KV (Grouped-Query Attention)
* **MoE Routing:** Top-4 expert routing with shared routing paths
* **Vocabulary Size:** 248,320 (Padded)
* **Context Length:** 131,072 tokens (Native) / 1M tokens (Extensible)
---
## Benchmark Results 📊🏆
| Benchmark | Setting | Metric | Qwen3.9-245B-A29B |
| --- | --- | --- | --- |
| **MMLU-Pro** | 5-shot | Accuracy | 83.2% |
| **MATH-500** | 0-shot | Accuracy (Thinking) | 95.1% |
| **AIME 2026** | Pass@1 | Accuracy | 89.4% |
| **LiveCodeBench** | 0-shot | Pass@1 | 71.3% |
| **GPQA Diamond** | 0-shot | Accuracy | 73.8% |
| **BFCL v3** | Multi-Turn | AST Match | 93.6% |
---
## Best Practices & Sampling Parameters ⚙️🎯
| Mode | Temperature | Top-P | Top-K | Presence Penalty | Max Reasoning Tokens |
| --- | --- | --- | --- | --- | --- |
| **Thinking Mode** | `1.0` | `0.95` | `20` | `0.0` | 262,144 |
| **Instruct Mode** | `0.7` | `0.80` | `20` | `1.5` | Disabled |
---
## Quickstart 🚀💻
### Transformers Inference
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "QwennAI/Qwen3.9-245B-A29B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
messages = [
{"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
{"role": "user", "content": "Prove that the square root of 2 is irrational."}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=True # Toggle thinking process
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=4096,
temperature=1.0,
top_p=0.95
)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
Deployment & Serving ⚡📦
vLLM
vllm serve QwennAI/Qwen3.9-245B-A29B \
--tensor-parallel-size 8 \
--enable-reasoning \
--reasoning-parser qwen3 \
--max-model-len 65536
SGLang
python -m sglang.launch_server \
--model-path QwennAI/Qwen3.9-245B-A29B \
--tp 8 \
--reasoning-parser qwen3 \
--port 8000
Citation 📜✍️
@article{qwen3.9,
title={Qwen3.9 Technical Report: Advancing Mixture-of-Experts Foundation Models},
author={Qwen Team},
journal={arXiv preprint arXiv:2608.xxxxx},
year={2026}
}