MiniCPM5-2B GGUF
Community GGUF quantizations of openbmb/MiniCPM5-2B.
About MiniCPM5-2B
MiniCPM5-2B is OpenBMB's dense 2B-class model for local assistants, coding
agents, tool-use workflows, reasoning, and resource-constrained deployment. It
uses the standard LlamaForCausalLM architecture, has 2,516,756,480 parameters,
42 layers, GQA with 16 query / 2 KV heads, and a native 131,072-token context
length. See the official model card
for the upstream capabilities, evaluation results, datasets, and supported
ecosystem.
This is a quantization-only release. No training, fine-tuning, merging, or weight modification other than BF16-to-GGUF conversion and quantization was performed. The source BF16 checkpoint was converted directly from the locked upstream revision; no quantized file was used as input to another quantization rung.
The upstream chat template is included as chat_template.jinja
and is also stored in the GGUF metadata. The official template supports
thinking and XML-style tool calls, but tool-call behavior is not certified by
this release's smoke test.
Fidelity measurements
The table below compares each published GGUF with the BF16 reference on a
held-out WikiText pilot: 16 chunks from wiki.test.raw and 16 chunks from
wiki.valid.raw, with a 4,096-token context and the same llama.cpp runtime.
Values are averaged across the two splits. Lower Mean KLD, ΔPPL, and RMS Δp,
and higher Top-1 agreement, indicate closer next-token behavior to BF16. The
BF16 reference mean PPL was 11.7502 in this pilot.
| File | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |
|---|---|---|---|---|
| MiniCPM5-2B-Q8_0.gguf | 0.001514 | 97.792% | +0.133% | 0.999% |
| MiniCPM5-2B-Q6_K.gguf | 0.006022 | 95.828% | +0.140% | 1.933% |
| MiniCPM5-2B-Q5_K_M.gguf | 0.018555 | 93.028% | +1.247% | 3.430% |
| MiniCPM5-2B-Q5_1.gguf | 0.018927 | 92.770% | +1.003% | 3.460% |
| MiniCPM5-2B-Q5_K_S.gguf | 0.021076 | 92.497% | +1.301% | 3.640% |
| MiniCPM5-2B-Q4_K_M.gguf | 0.063265 | 87.537% | +3.384% | 6.272% |
| MiniCPM5-2B-IQ4_NL.gguf | 0.072267 | 86.721% | +4.668% | 6.721% |
| MiniCPM5-2B-IQ4_XS.gguf | 0.074946 | 86.399% | +5.338% | 6.892% |
| MiniCPM5-2B-Q3_K_L.gguf | 0.160423 | 80.606% | +16.239% | 10.157% |
| MiniCPM5-2B-Q3_K_M.gguf | 0.182391 | 79.684% | +17.441% | 10.824% |
| MiniCPM5-2B-IQ3_M.gguf | 0.197723 | 78.853% | +15.540% | 11.770% |
| MiniCPM5-2B-IQ3_S.gguf | 0.229139 | 77.404% | +18.694% | 12.638% |
| MiniCPM5-2B-Q2_K.gguf | 0.741842 | 62.160% | +88.959% | 22.302% |
| MiniCPM5-2B-Q2_K_S.gguf | 1.054076 | 55.437% | +156.195% | 26.530% |
| MiniCPM5-2B-IQ2_XS.gguf | 2.119478 | 41.038% | +632.593% | 37.621% |
| MiniCPM5-2B-Q1_0.gguf | 13.416317 | 0.000% | +47,867,030% | 55.561% |
Q5_K_S is the compact modern Q5 option; Q5_1 is included as a
legacy-format comparison. Q6_K remains the Q6 control in this release.
The compact machine-readable results are available in
reproducibility/quality-summary.tsv,
with the corpus hashes, evaluation profile, and runtime commit recorded in the
reproducibility/manifest.md.
Quick start
With a recent llama.cpp build:
./llama-cli \
-m MiniCPM5-2B-Q4_K_M.gguf \
--chat-template-file chat_template.jinja \
--jinja \
--reasoning off \
-p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
-n 128 -c 4096 -ngl 99
The upstream model card documents vLLM, SGLang, Transformers, llama.cpp, Ollama, and LM Studio deployment paths. Use a runtime version that supports MiniCPM5 and the model's chat template.
Reproducibility and validation
The reproducibility/manifest.md locks the
upstream revision, source hashes, converter/runtime commit, calibration data,
imatrix hash, quantization commands, validation profile, and benchmark profile.
The package also includes the model-specific calibration corpus, combined
imatrix, quantization script, smoke-test script, and compact quality summary.
Published files passed load/generate smoke tests with tensor checking and the official MiniCPM5 chat template. Raw conversion, quantization, imatrix, smoke-test, quality-evaluation, and benchmark logs are retained locally and are not included here.
License and attribution
The upstream model is released under the Apache License 2.0. Preserve the
upstream attribution and LICENSE when redistributing these
derivative files. These are community GGUF quantizations, not an OpenBMB
release or endorsement.
Verify downloaded artifacts with SHA256SUMS.txt.