PaddleOCR-VL-1.6-MLX-8bit
MLX 8-bit quantization of PaddlePaddle/PaddleOCR-VL-1.6, a 0.9B-parameter vision-language model for document parsing. Converted for Apple Silicon, running locally with mlx-vlm.
Model details
- Base model: PaddlePaddle/PaddleOCR-VL-1.6 (0.9B, ERNIE-4.5-based)
- Quantization: 8-bit (MLX
mlx_vlm convert), group size 64, ~1.09 GB on disk - Architecture:
PaddleOCRVLForConditionalGeneration(custompaddleocr_vlmodel type; trust_remote_code required) - Context: up to 131,072 tokens
- Vision encoder: 14px patches, 784 px per merged token; resolution cap raised to 4,064,256 px (see below)
Resolution budget
This repo ships a higher vision resolution cap than upstream: max_pixels = 4,064,256 (upstream 1.6 ships 1,003,520; min_pixels stays 112,896). This is deliberate small-text OCR tuning — full pages are no longer downscaled below ~5,000 merged vision tokens. Under mlx-vlm 0.7.x the cap is read from preprocessor_config.json at load time; there is no per-request override, so select lower stages client-side by rendering/downscaling the image before sending it.
Caps are naturally expressed as k×k stages of the 28 px/token grid (28 = patch 14 × merge 2):
| Stage | max_pixels | Token budget | A4 render scale | Notes |
|---|---|---|---|---|
| 1×1 | 112,896 | 144 | 0.47 | = min_pixels |
| 2×2 | 451,584 | 576 | 0.94 | |
| 3×3 | 1,016,064 | 1,296 | 1.41 | ≈ upstream 1.6 default |
| 4×4 | 1,806,336 | 2,304 | 1.88 | |
| 5×5 | 2,822,400 | 3,600 | 2.35 | v1.0-era default |
| 6×6 (this repo) | 4,064,256 | 5,184 | 2.82 | A4 lands at ~5,100 tokens |
Token budget = (12k)² merged tokens; non-square images land slightly under budget (aspect-preserving resize on the factor-28 grid). Measured on Apple Silicon: full-A4 pages cost ~5× the wall time of the upstream cap (vision attention is global over unmerged patches); sub-cap images are unaffected. To revert to upstream behavior, set max_pixels back to 1003520 in preprocessor_config.json.
Capabilities
- Full-page OCR in Markdown and JSON formats
- Layout analysis, table recognition (structure), formula recognition (LaTeX)
- Chart and seal recognition, text spotting (detection + recognition)
Intended uses
- On-device document parsing on Apple Silicon (M-series)
- Local OCR pipelines where no data may leave the machine
- Served via
mlx_vlm.serveror the MLX Python API withtrust_remote_code=True
Usage
pip install mlx-vlm
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("OpenGryd/PaddleOCR-VL-1.6-MLX-8bit", trust_remote_code=True)
messages = apply_chat_template(
processor,
model.config,
prompt="OCR:",
num_images=1,
)
output = generate(model, processor, messages, image="media/document.png", max_tokens=4096)
print(output)
Note: with mlx-vlm 0.7.x, include the image via
apply_chat_template(...)(or use the server below) — a bare prompt string without an image placeholder produces an empty completion. Keep prompts exactlyOCR:-shaped: free-form suffixes can flip the model into spotting mode (inline<|LOC_n|>coordinates).
To serve OpenAI-compatible endpoints:
python -m mlx_vlm.server --model OpenGryd/PaddleOCR-VL-1.6-MLX-8bit --port 8090 --trust-remote-code
Limitations
- 8-bit quantization introduces a small accuracy delta vs. the bf16/fp16 base model
- Requires
trust_remote_code=True(custom modeling code ships with the repo) - No text-only inputs: image required
Evaluation
The base model achieves 96.3% on OmniDocBench v1.6, leading on text, formula, and table recognition. See the base model card for full benchmarks.
License
Apache-2.0 — inherited from the base model.
Citation
@misc{paddleocrvl2025,
title={PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training},
author={PaddlePaddle Team},
year={2025},
howpublished={\url{https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6}},
}