DeepSeek Coder 7B Instruct v1.5 — GGUF (full quant collection)

GGUF conversions of deepseek-ai/deepseek-coder-7b-instruct-v1.5 for llama.cpp, LM Studio, KoboldCPP, llamafile, and other GGUF runtimes.

This repo publishes 14 quantization files (K-quants, IQ-quants, and Edmon02’s Q8_0). Pick a file by quality vs VRAM; see the table below.

Available GGUF files

Exact byte sizes: see gguf-manifest.json on this repo.

BF16 / F16: Not stored here (~14 GB). Use upstream safetensors and llama.cpp convert + llama-quantize to build custom quants (e.g. Q4_0, Q5_0).

Provenance

Repository layout

deepseek-coder-7b-instruct-v1.5-GGUF/
├── README.md
├── gguf-manifest.json
├── .gitattributes
├── deepseek-coder-7b-instruct-v1.5.Q2_K.gguf
├── deepseek-coder-7b-instruct-v1.5.Q3_K_S.gguf
├── … (all quant variants)
└── deepseek-coder-7b-instruct-v1.5.Q8_0.gguf

Download one quant

pip install -U huggingface_hub

# Example: best balance (Q4_K_M)
huggingface-cli download Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF \
  deepseek-coder-7b-instruct-v1.5.Q4_K_M.gguf \
  --local-dir ./models/deepseek-7b-q4km

# Example: maximum quality (Q8_0, Edmon02)
huggingface-cli download Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF \
  deepseek-coder-7b-instruct-v1.5.Q8_0.gguf \
  --local-dir ./models/deepseek-7b-q8

Download everything (≈50 GB total):

huggingface-cli download Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF \
  --local-dir ./models/deepseek-7b-all-quants

Python:

from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF",
    filename="deepseek-coder-7b-instruct-v1.5.Q4_K_M.gguf",
)

Quick start (llama.cpp)

MODEL=deepseek-coder-7b-instruct-v1.5.Q4_K_M.gguf   # or any quant above

./llama-cli -m "$MODEL" \
  -p "### Instruction:\nWrite a binary search in Python.\n\n### Response:\n" \
  -n 512 -c 4096 --temp 0.1

./llama-server -m "$MODEL" -c 4096 --port 8080

Chat template (instruct v1.5)

### Instruction:
{user message}

### Response:

See upstream chat_template for the full Jinja definition.

Choosing a quant

Lower quants run faster but lose syntax fidelity on long code; benchmark on your prompts.

Intended uses

• Local code assistants (IDE, CLI, agents)
• Offline development without API keys
• Comparing quant trade-offs on the same Armenian/English coding stack

Out of scope

• Non-code prompts (model often refuses by design)
• Fine-tuning from GGUF (use upstream safetensors)
• Guaranteed parity with DeepSeek cloud APIs

Limitations

• IQ/K quants from mradermacher may differ slightly from Edmon02 Q8_0
• 4K context only
• ~50 GB if you download all files — use one quant in production

Maintainer tooling

# Upload missing quants + refresh manifest (from workspace)
python scripts/sync_deepseek_gguf_quants.py
python scripts/sync_deepseek_gguf_quants.py --remove-legacy   # drop old unnamed .gguf
python scripts/push_model_cards.py --only gguf

Citation

@misc{deepseek_coder_7b_v15_gguf,
  author = {Avetisyan, Edmon},
  title = {DeepSeek Coder 7B Instruct v1.5 (GGUF full quant collection)},
  year = {2024},
  howpublished = {\url{https://huggingface.co/Edmon02/deepseek-coder-7b-instruct-v1.5-GGUF}}
}

License

DeepSeek Model License. Third-party quants retain the same upstream terms; verify before commercial redistribution.