Ministral-3-Instruct-2512-AIO-GGUF

The Ministral 3 Instruct models (3B, 8B, and 14B variants from mistralai) form a family of compact, edge-optimized vision-language models designed for efficient deployment on resource-constrained hardware, featuring FP8 precision for the Instruct versions to fit in as little as 8GB VRAM (3B), 12GB (8B), or higher for 14B while supporting a 256k token context window. Comprising a language model (3.4B, 8.4B, or larger parameters) paired with a 0.4B vision encoder, they excel in multimodal tasks like image analysis, captioning, document understanding, multilingual support across dozens of languages (English, French, Spanish, Chinese, etc.), strong system prompt adherence, agentic function calling with JSON output, and real-time applications such as translation, data extraction, and short content generation under Apache 2.0 license. Benchmarks highlight competitive reasoning (e.g., Ministral 3 3B scores 0.721 on AIME25, 0.534 on GPQA Diamond) and instruct performance (0.305 Arena Hard, 7.83 MM MTBench for 3B), outperforming peers like Qwen3-VL-2B in efficiency for local/embedded use cases with vLLM or Transformers integration.

Quantized GGUF for the following Ministral-3 models

+ mistralai/ministral-3-14b-instruct-2512
+ mistralai/ministral-3-8b-instruct-2512
+ mistralai/ministral-3-3b-instruct-2512
+ mistralai/ministral-3-14b-instruct-2512-bf16
+ mistralai/ministral-3-8b-instruct-2512-bf16
+ mistralai/ministral-3-3b-instruct-2512-bf16
  
Download from the respective file links below or from the Files and Versions section.

Ministral-3-14B-Instruct-2512 [GGUF]

Ministral-3-8B-Instruct-2512 [GGUF]

Ministral-3-3B-Instruct-2512 [GGUF]

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):