DeepSeek-R1-Llama-8B-F32-GGUF
DeepSeek-R1-Llama-8B-F32-GGUF is a quantized version of DeepSeek-R1-Distill-Llama-8B, trained using reinforcement learning (RL) directly on the base model without relying on supervised fine-tuning (SFT) as a preliminary step. This approach enables the model to explore chain-of-thought (CoT) reasoning for solving complex problems, leading to the development of DeepSeek-R1-Zero. DeepSeek-R1-Zero demonstrates capabilities such as self-verification, reflection, and generating extended chains of thought
Model File
| File Name | Size | Format | Notes |
|---|---|---|---|
| DeepSeek-R1-Llama-8B.BF16.gguf | 15.6 GB | GGUF | BF16 precision model |
| DeepSeek-R1-Llama-8B.F16.gguf | 16.1 GB | GGUF | FP16 precision model |
| DeepSeek-R1-Llama-8B.F32.gguf | 32.1 GB | GGUF | FP32 precision model |
| .gitattributes | 1.75 kB | Text | Git LFS tracking config |
| config.json | 31 B | JSON | Model configuration file |
| README.md | 767 B | Markdown | This readme file |
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
