gpt-oss-120b-PS-GGUF
GGUF quantizations of gpt-oss-120b by PeasantSmith & BlackFrost AI.
| Quantization | BPW | Size | Quantizer |
|---|---|---|---|
| IQ4_NL | 4.91 | 71.75GB | Peasant Smith |
| IQ2_XXS | 2.x | 66.1GB | BlackFrost AI |
Testing
Raw perplexity doesn't work here: gpt-oss needs the harmony response format, and even the F16 source returns NaN on unformatted text past chunk 8. Validated with task-based tests instead: Quantization quality was validated with task-based tests instead:
| Test | Result |
|---|---|
| Greedy generation (factual question) | PASS, correct answer + complete reasoning block |
| LRU cache implementation task (8192 ctx) | PASS, doubly-linked-list, O(1) get/put, coherent |
| Needle recall (magic value 9137@8k ctx) | PASS, exact answer: 9137 |
| Runtime errors | 0 |
Recommended parameters
- Temperature:
1.0 - Top-k:
0(i.e. all) - Top-p:
1.0
Example llama.cpp invocation:
llama-cli -m gpt-oss-120b-IQ4_NL.gguf \
-c 8192 -fa -ngl 99 -ncmoe 36 \
--split-mode layer --tensor-split 1,1,1 \
--temp 1.0 --top-k 0 --top-p 1.0
Flag names vary across llama.cpp builds.
Reproduce/Verify
llama-quantize \
--tensor-type "ffn_gate_exps=iq4_nl" \
--tensor-type "ffn_up_exps=iq4_nl" \
--tensor-type "ffn_down_exps=q5_0" \
--token-embedding-type q8_0 \
gpt-oss-120b-F16.gguf gpt-oss-120b-IQ4_NL.gguf q8_0
License
Quantized derivative of gpt-oss-120b: license and usage terms follow upstream, verify before redistribution. Base model by OpenAI.
Additional information
The quantization is released under the PeasantSmith (PS) signature, reflected in the repository name gpt-oss-120b-PS-GGUF. Individual GGUF files use the standard model-<quant>.gguf naming (e.g. gpt-oss-120b-IQ4_NL.gguf) so Ollama and HuggingFace quant-tag parsing work correctly. The IQ4_NL label identifies this specific quantized release and should not be confused with the upstream model's original naming.