Parakon 30B
Qwen3-30B-A3B-Instruct-2507 compressed by Parakon's pipeline: 61 GB (fp16) → 5.94 GB, a 10.3× reduction, with 88% median quality retention on a six-benchmark suite.
Quality — paired, same harness
Each figure is the compressed model's score as a share of the same checkpoint at full precision, run through the same harness (greedy decoding, one configuration). Retention holds on knowledge, tool use and reasoning; it thins on code and strict instruction formatting — reported at the same size as the rest.
| Benchmark | Retention |
|---|---|
| MuSR (reasoning) | 93% |
| GSM8K (math) | 90% |
| MMLU-Redux (knowledge) | 88% |
| BFCL-v3 (tool calling) | 88% |
| HumanEval+ (code) | 77% |
| IFEval, prompt-strict (instruction following) | 76% |
| Median | 88% |
Speed — measured
| Hardware | Throughput |
|---|---|
| 1× NVIDIA A10G, 24 GB (CUDA, fused decode) | 162.9 tokens/s |
| M3 MacBook Air, 16 GB (Metal) | 12.7 tokens/s |
| CPU only, 48 vCPU | 7.7 tokens/s |
Peak VRAM on the A10G run: 6.5 GB. The weights execute directly from the 1-bit blocks; no dequantized copy is ever materialized.
The published runtime release is CUDA. The Metal and CPU numbers above were measured with internal builds that are not published yet.
Recommended settings
Chat template: Qwen3 (ChatML). Native context length: up to 262,144 tokens.
General-purpose sampling:
| Parameter | Value |
|---|---|
| temperature | 0.7 |
| top_p | 0.8 |
| top_k | 20 |
| min_p | 0 |
Runtime
This artifact uses Parakon's own storage format and requires the Parakon runtime to execute. The CUDA runtime is public: Parakon/parakon-runtime — a fork of llama.cpp with Parakon's kernels, with build and run instructions in its README.
License & access
Released under the Parakon Community License:
- Always free — research, personal use, evaluation, and benchmarking (publishing your results is encouraged, never restricted)
- Free commercial use for organizations under 100 employees and $1M annual revenue — production included
- Larger organizations need a commercial agreement — contact the Parakon team through this organization's page
- No re-hosting — refer others to this repository for the weights
The runtime is licensed separately. For deployment licensing, runtime access, or compression engagements on your own models: get in touch.