NuExtract3-GGUF

NuExtract3 is NuMind's 4-billion-parameter unified vision-language model built on Qwen3.5-4B, designed for document understanding by combining strong structured information extraction with high-quality image-to-Markdown conversion, making it suitable for extraction pipelines, OCR, and RAG preprocessing across scans, receipts, forms, invoices, contracts, and tables. It accepts text, images, or both alongside a JSON template describing the desired output schema (with typed leaves like verbatim-string, date-time, currency, enums, and multi-enums), optional natural-language instructions, and optional in-context examples, and can also generate extraction templates from plain-language descriptions or convert documents directly into structured Markdown (with HTML tables and LaTeX math) — all while supporting both fast non-reasoning and deeper thinking inference modes. On NuMind's internal structured-extraction benchmark of ~600 diverse documents, NuExtract3.4-4B-RL achieves the top average score (0.651) with far fewer failed/non-deserializable outputs than larger models like Qwen3.5-9B, outperforming Gemma-4-E4B-it, GLM-4.6V-Flash, and Nemotron-3-Nano-Omni, with the paper noting that heavier reasoning actually hurts smaller models by triggering repetition loops and output-length failures. The model is deployable via vLLM with an OpenAI-compatible API supporting Multi-Token Prediction speculative decoding for faster throughput, or directly through Hugging Face Transformers, and is released under the Apache 2.0 license.

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp