Qwen3-4B — W1.88, QAD + OPD
Qwen3-4B quantized to 1.88 bits per weight, then recovered with quantization-aware distillation followed by on-policy distillation. Quantization is baked in: load it and evaluate it.
Code, evaluation harness and the training recipe: MingZwhy/QAOPD.
Results
Measured through this checkpoint, against the unquantized model.
| GSM8K | MATH-500 | AMC23 | MBPP | HumanEval | QA9 | |
|---|---|---|---|---|---|---|
| this model | 64.59 | 36.00 | 16.25 | 48.7 | 60.4 | 52.83 |
| Qwen3-4B BF16 | 86.35 | 68.80 | 55.00 | 67.6 | 82.3 | 61.63 |
GSM8K 5-shot strict-match, MATH-500 4-shot, AMC23 avg@16, MBPP and HumanEval pass@1 greedy, QA9 the equal-weight mean of nine likelihood-scored benchmarks.
Quantization
| weights | mixed INT1.58 / INT4 in blocks of 256, 12.5% of blocks at INT4 → 1.88 bits effective |
| embedding, output head | INT4 |
| activations | INT8 |
| KV cache | 16-bit |
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "MingZwhy/Qwen3-4B-W1.88-QAOPD"
model = AutoModelForCausalLM.from_pretrained(name, trust_remote_code=True)
tok = AutoTokenizer.from_pretrained(name)
To reproduce the numbers above with the same harness, see docs/EVALUATION.md.
License
Apache-2.0, inherited from Qwen3-4B.