Qwen3.5-2B for RK3588 NPU
This repository provides a hardware-accelerated port of Qwen3.5-2B optimized for Rockchip RK3588 NPU.
User:<image>Describe the image.
Answer: This is a surreal, digitally manipulated image that blends science fiction with everyday life in an unexpected setting. Here’s a detailed description:
Main Subject:
- A person wearing a white astronaut suit reclines on what appears to be a lunar surface or spacecraft interior.
- Their legs are propped up against the edge of a green cooler box, which has faint graffiti-style writing (“SOMETHING”) visible on its side.
- In their right hand, they hold a bright green glass beer bottle — likely a craft beer — as if enjoying a drink in zero gravity.
Setting & Background:
- The environment is clearly extraterrestrial: gray, cratered terrain resembling the Moon’s surface.
- Behind the astronaut, Earth dominates the sky — its blue oceans and swirling white clouds are visible against the black void of space dotted with stars.
- To the right, part of a metallic ladder or structure leans into the frame, suggesting this might be inside a spacecraft or on a lunar module.
Atmosphere & Style:
- The lighting is dramatic — strong highlights on the astronaut’s helmet visor and suit, contrasting sharply with the dark background.
- There’s a sense of irony or absurdity: an astronaut relaxing with beer next to Earth, defying gravity and normal human behavior in space.
- The image has a hyper-realistic yet dreamlike quality, enhanced by digital effects like lens flares and atmospheric glow around Earth.
Overall Impression: It’s a humorous, thought-provoking piece that plays on the contrast between isolation in space and comfort found in mundane pleasures — drinking beer while floating above our own planet. It invites reflection on human resilience, curiosity, and the strange ways we adapt to extreme environments.
Model Files
| Component | File | Precision |
|---|---|---|
| LLM | qwen3.5-2b_w8a8_rk3588.rkllm | W8A8 |
| Vision Encoder | qwen3.5-2b-vision_rk3588.rknn | FP16 |
Hardware Requirements
- Rockchip RK3588 / RK3588S
- RKNPU2 driver
- Tested on:
- Rock 5C
- Ubuntu 22.04 / 24.04 (Joshua Riek)
Runtime Requirements
- RKLLM runtime (version 1.3.0)
- RKNN runtime (rknpu2)
- OpenCV (for image preprocessing)
Model performance benchmark (FPS)
All models, with C++ examples, can be found on the Q-engineering GitHub. All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.
| model | RAM (GB)1 | llm cold sec2 | llm warm sec3 | vlm cold sec2 | vlm warm sec3 | Resolution | Tokens/s |
|---|---|---|---|---|---|---|---|
| Qwen3.5-9B | 9.2 | 97.1 | 97.1 | 11.5 | 11.5 | 448 x 448 | 3.2 |
| Qwen3.5-4B | 5.4 | 52.8 | 6.2 | 8.5 | 0.9 | 448 x 448 | 5.2 |
| Qwen3.5-2B | 2.9 | 23.9 | 3.2 | 8.5 | 0.8 | 448 x 448 | 11.0 |
| Qwen3.5-0.8B | 1.3 | 10.6 | 1.9 | 2.7 | 0.2 | 448 x 448 | 21.6 |
| Qwen3-2B | 3.1 | 21.9 | 2.6 | 10.0 | 0.9 | 448 x 448 | 11.5 |
| Qwen3-4B | 8.7 | 49.6 | 5.6 | 10.6 | 1.1 | 448 x 448 | 5.7 |
| InternVL3.5-1B | 1.9 | 8.3 | 8.0 | 1.5 | 0.8 | 448 x 448 | 24 |
| InternVL3.5-2B | 3.0 | 22 | 8.0 | 2.7 | 0.8 | 448 x 448 | 11.2 |
| InternVL3.5-4B | 5.4 | 50 | 8.0 | 5.9 | 0.8 | 448 x 448 | 5 |
| InternVL3.5-8B | 8.8 | 92 | 8.0 | 50.5 | 5.8 | 448 x 448 | 3.5 |
| Qwen2.5-3B | 4.8 | 48.3 | 4.0 | 17.9 | 1.8 | 392 x 392 | 7.0 |
| Qwen2-7B | 8.7 | 86.6 | 34.5 | 37.1 | 20.7 | 392 x 392 | 3.7 |
| Qwen2-2.2B | 3.3 | 29.1 | 2.5 | 17.1 | 1.7 | 392 x 392 | 12.5 |
| InternVL3-1B | 1.3 | 6.8 | 1.1 | 7.8 | 0.75 | 448 x 448 | 30 |
| SmolVLM2-2.2B | 3.4 | 21.2 | 2.6 | 10.5 | 0.9 | 384 x 384 | 11 |
| SmolVLM2-500M | 0.8 | 4.8 | 0.7 | 2.5 | 0.25 | 384 x 384 | 31 |
| SmolVLM2-256M | 0.5 | 1.1 | 0.4 | 2.5 | 0.25 | 384 x 384 | 54 |
1 The total used memory; LLM plus the VLM. 2 When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start. The duration depends on your OS, I/O transfer rate, and memory mapping. 3 Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.
Example Usage
Notes
- KV Cache up to 16384
- This is not a Transformers-compatible model
- This repository provides precompiled NPU binaries
- CPU fallback is not supported