FlashNext-MoE-9B-A1B
Randomly initialized, language-only Qwen4ExpForCausalLM checkpoint.
- Parameters: 9,000,082,560 (9.000082560B)
- Approximate active backbone parameters per token: 0.995B
- 32 layers: 24 Gated DeltaNet and 8 Qwen Sparse Attention
- 64 routed experts, top-2, plus one shared expert per layer
- No vision tower and no MTP head
- BF16 weights in five safetensors shards
- Qwen3.8-Flash-Next tokenizer and chat template
These weights are random and must be pretrained or distilled before useful inference.
from transformers import AutoModelForCausalLM, AutoTokenizer
path = "/path/to/flashnext-moe-9b/model"
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, dtype="bfloat16")