gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-GGUF
This repository contains GGUF format model files for Ayodele01's gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill.
These models were compiled and quantized via llama.cpp to enable efficient local inference on consumer hardware.
Available Quantizations
| File Name | Description |
|---|---|
gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q8_0.gguf | 8-bit quantization. Near unquantized performance, largest file size. |
gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q6_K.gguf | 6-bit quantization. Very high quality, minimal degradation from original. |
gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q5_K_M.gguf | 5-bit quantization. Higher quality, slightly larger size and slower inference. |
gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q4_K_M.gguf | 4-bit quantization. Recommended. Excellent balance of speed, memory usage, and quality. |
gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q3_K_M.gguf | 3-bit quantization. Very high compression, fast inference, lower quality. |