Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled NVFP4 GGUF
GGUF quantization of lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — a 35B MoE (3B active) reasoning model distilled from Claude 4.7 Opus.
Model Details
- Architecture: Qwen3.5 MoE, 41 blocks (40 layers + 1 MTP head), 256 experts (8 active/token)
- Active Parameters: ~3B
- Context: 262,144 tokens
- Source: Apache 2.0 license
- Text-only: No vision support
Quantization
| File | Type | Size | BPW |
|---|---|---|---|
qwen36-35b-distill-nvfp4.gguf | NVFP4 (all tensors) | ~18.81 GB | 4.55 |
All weight tensors quantized to NVFP4 (4-bit floating point). F32/F16 bias and norm tensors kept at original precision.
Usage
./llama-server -m qwen36-35b-distill-nvfp4.gguf --host 0.0.0.0 --port 8080
Credits
- Original model: lordx64
- Quantization: FreedomAISVR