MiniCPM5-2B for RKNN3 / RKLLM3 Server

RKNN3-converted openbmb/MiniCPM5-2B LLM artifacts for the Rockchip RK1820 and RK1828 AI coprocessors, ready for deployment with Rockchip's RKLLM3 Server (rkllm3-server) or the RKNN3 Runtime examples.

These are hardware-specific deployment artifacts, not a Transformers checkpoint and not an RKLLM model. They were produced with rknn3-toolkit v1.1.0 for the new RKNN3 toolchain and are intended for use with a compatible RKNN3 Runtime, firmware, demo, or rkllm3-server.

Base model

• Upstream model: openbmb/MiniCPM5-2B
• Model type: causal language model (LlamaForCausalLM)
• Parameters: 2.52B total, 1.98B excluding embeddings
• Upstream context length: 131,072 tokens
• Upstream license: Apache-2.0

The usable context length of an RKNN3 deployment is determined by the converted model and runtime configuration. Do not assume that it is identical to the upstream checkpoint's maximum context length.

Conversion

RKNN3 is a separate toolchain from RKLLM Toolkit, RKNN-Toolkit, and RKNN-Toolkit2. Use RKNN3 v1.1.0 runtime components and RK182x firmware that are compatible with these converted artifacts.

Files

.
├── MiniCPM5-2B-w4a16.weight
├── MiniCPM5-2B.embed.bin
├── MiniCPM5-2B.tokenizer.gguf
├── RK1820/
│   └── MiniCPM5-2B-RK1820-w4a16.rknn
└── RK1828/
    └── MiniCPM5-2B-RK1828-w4a16.rknn

The tokenizer, FP16 embedding table, and W4A16 weight file are shared by both targets. The RK1820 and RK1828 conversions produced byte-identical .weight files, so this repository stores one copy at the root. The .rknn files differ and you must select the one matching your coprocessor.

Keep all four files from this repository revision together. A model graph or weight from a different conversion may be incompatible even when its filename looks similar.

Download

Install the current Hugging Face CLI and download the complete repository:

hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  --local-dir MiniCPM5-2B-RKNN3

To download only the RK1820 runtime set:

hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  MiniCPM5-2B-w4a16.weight \
  MiniCPM5-2B.tokenizer.gguf \
  MiniCPM5-2B.embed.bin \
  --local-dir MiniCPM5-2B-RKNN3

Replace RK1820 with RK1828 and use the corresponding filename for an RK1828 deployment.

Run with rkllm3-server

The official rkllm3-server accepts the graph, separated weight, tokenizer, and embedding as explicit paths. For RK1820:

/usr/bin/rkllm3-server \
  -m MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  --weight MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  --vocab MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  --embed MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin

For RK1828, change only the graph path:

-m MiniCPM5-2B-RKNN3/RK1828/MiniCPM5-2B-RK1828-w4a16.rknn

Use the server binary, RKNN3 Runtime libraries, transfer proxy, and RK182x firmware supplied for the same compatible RKNN3 release. RK1820/RK1828 operate as coprocessors and require a supported host platform and connection method.

Run the rknn3-model-zoo C++ demo

Build and deploy the official MiniCPM5 example as described by the rknn3-model-zoo. From the deployed demo directory, an RK1820 invocation is:

export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH

./rknn_minicpm5_demo \
  /path/to/MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin \
  0xff \
  "Explain the theory of relativity"

For RK1828, select the graph under RK1828/. The 0xff argument is the NPU core mask used in Rockchip's example; adjust it for your deployment when necessary.

Compatibility and limitations

• These files require the RKNN3 software stack. They cannot be loaded by Transformers, llama.cpp, the legacy RKLLM Runtime, RKNN-Toolkit, or RKNN-Toolkit2.
• The .rknn graph is target-specific. Do not use the RK1820 graph on RK1828 or the RK1828 graph on RK1820.
• The root .weight, .tokenizer.gguf, and .embed.bin files are shared by the two graphs in this repository revision.
• Quantization can change generation quality relative to the upstream BF16 model. Validate accuracy and performance for your application.
• Runtime behavior depends on the installed RKNN3 Runtime, host software, transfer method, and RK182x firmware.

References and acknowledgements

• MiniCPM5-2B by OpenBMB
• Rockchip rknn3-toolkit
• Rockchip rknn3-model-zoo
• MiniCPM5 conversion and deployment example

Thanks to OpenBMB, Rockchip, and the RKNN community.