Olmo-3-7B-Instruct-AIO-GGUF
Olmo-3-7B-Instruct is a 7-billion-parameter transformer-based autoregressive language model from the Allen Institute for AI, fine-tuned with supervised fine-tuning (SFT) and direct preference optimization (DPO) on datasets like Tulu 2 and UltraFeedback for enhanced question answering and instruction following. It supports a context length of 2048 tokens and excels in generating high-quality, context-aware, and safe text responses across various NLP tasks including multi-turn chat, tool use, and detailed instructions. Designed as a research model, it aims to advance understanding of language models with transparency on training and safety considerations, balancing efficient inference with strong performance, making it suitable for applications requiring precise adherence to instructions in domains like technical, educational, and legal fields. The model and its training code are openly available under the Apache 2.0 license.
Olmo-3-7B-Instruct [GGUF]
| File Name | Quant Type | File Size | File Link |
|---|---|---|---|
| Olmo-3-7B-Instruct.BF16.gguf | BF16 | 14.6 GB | Download |
| Olmo-3-7B-Instruct.F16.gguf | F16 | 14.6 GB | Download |
| Olmo-3-7B-Instruct.F32.gguf | F32 | 29.2 GB | Download |
| Olmo-3-7B-Instruct.IQ4_XS.gguf | IQ4_XS | 4.03 GB | Download |
| Olmo-3-7B-Instruct.Q2_K.gguf | Q2_K | 2.86 GB | Download |
| Olmo-3-7B-Instruct.Q3_K_L.gguf | Q3_K_L | 3.95 GB | Download |
| Olmo-3-7B-Instruct.Q3_K_M.gguf | Q3_K_M | 3.65 GB | Download |
| Olmo-3-7B-Instruct.Q3_K_S.gguf | Q3_K_S | 3.3 GB | Download |
| Olmo-3-7B-Instruct.Q4_K_M.gguf | Q4_K_M | 4.47 GB | Download |
| Olmo-3-7B-Instruct.Q4_K_S.gguf | Q4_K_S | 4.25 GB | Download |
| Olmo-3-7B-Instruct.Q5_K_M.gguf | Q5_K_M | 5.21 GB | Download |
| Olmo-3-7B-Instruct.Q5_K_S.gguf | Q5_K_S | 5.08 GB | Download |
| Olmo-3-7B-Instruct.Q6_K.gguf | Q6_K | 5.99 GB | Download |
| Olmo-3-7B-Instruct.Q8_0.gguf | Q8_0 | 7.76 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ1_M.gguf | i1-IQ1_M | 1.94 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ1_S.gguf | i1-IQ1_S | 1.82 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ2_M.gguf | i1-IQ2_M | 2.68 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ2_S.gguf | i1-IQ2_S | 2.51 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ2_XS.gguf | i1-IQ2_XS | 2.32 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ2_XXS.gguf | i1-IQ2_XXS | 2.14 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ3_M.gguf | i1-IQ3_M | 3.47 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ3_S.gguf | i1-IQ3_S | 3.3 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ3_XS.gguf | i1-IQ3_XS | 3.15 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ3_XXS.gguf | i1-IQ3_XXS | 2.9 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ4_NL.gguf | i1-IQ4_NL | 4.22 GB | Download |
| Olmo-3-7B-Instruct.i1-IQ4_XS.gguf | i1-IQ4_XS | 4 GB | Download |
| Olmo-3-7B-Instruct.i1-Q2_K.gguf | i1-Q2_K | 2.86 GB | Download |
| Olmo-3-7B-Instruct.i1-Q2_K_S.gguf | i1-Q2_K_S | 2.64 GB | Download |
| Olmo-3-7B-Instruct.i1-Q3_K_L.gguf | i1-Q3_K_L | 3.95 GB | Download |
| Olmo-3-7B-Instruct.i1-Q3_K_M.gguf | i1-Q3_K_M | 3.65 GB | Download |
| Olmo-3-7B-Instruct.i1-Q3_K_S.gguf | i1-Q3_K_S | 3.3 GB | Download |
| Olmo-3-7B-Instruct.i1-Q4_0.gguf | i1-Q4_0 | 4.23 GB | Download |
| Olmo-3-7B-Instruct.i1-Q4_1.gguf | i1-Q4_1 | 4.65 GB | Download |
| Olmo-3-7B-Instruct.i1-Q4_K_M.gguf | i1-Q4_K_M | 4.47 GB | Download |
| Olmo-3-7B-Instruct.i1-Q4_K_S.gguf | i1-Q4_K_S | 4.25 GB | Download |
| Olmo-3-7B-Instruct.i1-Q5_K_M.gguf | i1-Q5_K_M | 5.21 GB | Download |
| Olmo-3-7B-Instruct.i1-Q5_K_S.gguf | i1-Q5_K_S | 5.08 GB | Download |
| Olmo-3-7B-Instruct.i1-Q6_K.gguf | i1-Q6_K | 5.99 GB | Download |
| Olmo-3-7B-Instruct.imatrix.gguf | imatrix | 4.59 MB | Download |
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
