MiniCPM5-2B GGUF
GGUF quantizations of OpenBMB/MiniCPM5-2B, a 2B dense Llama-based model for local deployment, coding, reasoning, long-context work, and tool use. The source checkpoint supports a native context length of 131,072 tokens (128K).
Benchmarks
Benchmark results reported by OpenBMB for the original MiniCPM5-2B checkpoint.
Capability comparison reported by OpenBMB for the original MiniCPM5-2B checkpoint.
GGUF files
| Quantization | File | Size |
|---|---|---|
| Q2_K | MiniCPM5-2B-Q2_K.gguf | 1.04 GB |
| Q3_K_M | MiniCPM5-2B-Q3_K_M.gguf | 1.29 GB |
| Q4_0 | MiniCPM5-2B-Q4_0.gguf | 1.49 GB |
| Q4_K_S | MiniCPM5-2B-Q4_K_S.gguf | 1.50 GB |
| Q4_K_M | MiniCPM5-2B-Q4_K_M.gguf | 1.56 GB |
| Q5_K_M | MiniCPM5-2B-Q5_K_M.gguf | 1.81 GB |
| Q6_K | MiniCPM5-2B-Q6_K.gguf | 2.07 GB |
| Q8_0 | MiniCPM5-2B-Q8_0.gguf | 2.68 GB |
| IQ2_M | MiniCPM5-2B-IQ2_M.gguf | 0.97 GB |
| IQ3_M | MiniCPM5-2B-IQ3_M.gguf | 1.23 GB |
| IQ4_XS | MiniCPM5-2B-IQ4_XS.gguf | 1.42 GB |
The model is text-only. No vision projector or MTP files are included. The IQ files use an importance matrix generated from WikiText-2 and are intended for recent llama.cpp builds. SHA-256 checksums are provided in SHA256SUMS.txt.
Chat template
The GGUF files embed the upstream chat template. chat_template.jinja is provided as an external copy for runtimes that require a separate template file.
Usage
Use a current llama.cpp build with MiniCPM5 support. The example below uses an 8K context; increase -c up to 131072 when sufficient memory is available.
llama-cli \
-m MiniCPM5-2B-Q4_K_M.gguf \
-c 8192 --jinja \
--temp 1.0 --top-p 0.95 \
-p "Explain why reproducible builds matter."
For an OpenAI-compatible server:
llama-server \
-m MiniCPM5-2B-Q4_K_M.gguf \
-c 8192 --jinja --host 0.0.0.0 --port 8080
Tool-call behavior depends on the serving runtime's parser and API integration; use the embedded template and verify tool calls in the target application.
Source
- Model: OpenBMB/MiniCPM5-2B
- Source revision:
3497c460c89e00520c3cfa2e73f49ab7647f1177 - Conversion: the original Q4_0–Q8_0 files use upstream llama.cpp commit
f114f91f9ed6792cf402437e3874adad98902744; the additional Q2_K, Q3_K_M, Q4_K_S, IQ2_M, IQ3_M, and IQ4_XS files use upstream commit67672dc5b76f8bc17785a19d3dc6d1463fc2902c - License: Apache-2.0
- Checksums: SHA256SUMS.txt