DeepSeek-R1-Distill-Qwen-1.5B (IQ2_XXS GGUF)
A heavily compressed GGUF quantization of DeepSeek-R1-Distill-Qwen-1.5B — the smallest official DeepSeek reasoning model.
This card covers the single file:
| Property | Value |
|---|---|
| File | DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf |
| Size | 560.37 MiB (587.59 MB) |
| Quantization | IQ2_XXS (≈ 2.06 bits/weight) |
| Parameters | 1.78B (total) / ~1.5B (active) |
| Architecture | Qwen2 |
| Context length | 32768 tokens |
| GGUF version | 3 (latest) |
Why this quantization
IQ2_XXS is the smallest IQ type that keeps the model usable on low-VRAM
hardware. At ~0.56 GB the weights load on any 4 GB-class GPU (e.g. GTX 1650)
while retaining the model's reasoning behaviour, including the
[Start thinking] / [End thinking] chain-of-thought format.
It was produced from the F16 source with llama.cpp using an importance
matrix computed with llama-imatrix over a wikitext-2 calibration set. The
advertised context was capped at 32768 tokens so the KV cache fits a 4 GB GPU
when fully offloaded (-ngl 999).
Usage
llama.cpp (CLI)
llama-cli \
-m DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
-p "What is the capital of France?" \
-n 256 \
-ngl 999 \
-c 32768
llama-server (OpenAI-compatible API)
llama-server \
-m DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
-ngl 999 \
-c 32768 \
--port 8080
# curl http://localhost:8080/v1/chat/completions ...
Docker Model Runner (docker model)
docker model package \
--gguf DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
--context-size 32768 \
deepseek-r1
docker model run deepseek-r1 "What is the capital of France?"
Ollama
ollama create deepseek-r1-iq2-xxs -f Modelfile # FROM ./DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf
ollama run deepseek-r1-iq2-xxs
Quality expectations
IQ2_XXS is an aggressive compression. The model answers factual questions
correctly and keeps its reasoning structure, but expect degraded fluency and
precision compared to F16 / Q4_K_M. For higher quality on the same hardware,
prefer Q4_K_M (~1.0 GB); for absolute minimum size, IQ1_S or Q1_0 exist
but degrade much faster.
Credits & license
- Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (MIT)
- Source GGUF: bartowski/DeepSeek-R1-Distill-Qwen-1.5B-GGUF (MIT)
- Quantized with llama.cpp +
llama-imatrix - This file is released under the MIT license.