IndicOCR MLX 4-bit (IndicBlockOCR)

This repository provides an optimized 4-bit quantized MLX export of IndicBlockOCR (Stage 2 of bodhan-ai/indic-ocr), tailored for real-time document transcription on Apple Silicon Metal GPUs (M1/M2/M3/M4) with minimal memory footprint.

The recognizer architecture is based on Qwen3.5-0.8B with a 151k-token Sarvam-30B vocabulary covering 22 official Indic scripts, English, numbers, and mathematical notation.

Precision Variants

• hari31416/indic-ocr-mlx-4bit (This repository): 636 MB model weights, 391 ms/crop (24.96x speedup), 4.83% CER.
• hari31416/indic-ocr-mlx-8bit: 1.0 GB model weights, 486 ms/crop (20.09x speedup), 1.56% CER.
• hari31416/indic-ocr-mlx-bf16: 1.6 GB model weights, 683 ms/crop (14.30x speedup), 1.69% CER.

Empirical Parity and Accuracy Benchmarks

Evaluated against the PyTorch reference baseline on 50 real-world Indic document crops from ai4bharat/indicdlp.

Stage 2 Recognition Parity (4-bit vs PyTorch Baseline)

Stage 1 Layout Detection (PP-DocLayoutV3 on PyTorch MPS)

Stage 1 runs with PyTorch on Apple's MPS backend. The recognizer uses MLX, but the layout detector does not: this project has no MLX PP-DocLayoutV3 runtime, and converting the layout tensors alone does not improve inference.

Understanding Exact Match vs Character Error Rate

While the Exact Match rate is 60.0%, the mean Character Error Rate is only 4.83% and 82.0% of crops have less than 5% CER. Exact Match is an all-or-nothing metric: on long paragraph crops (e.g. 600+ characters), a single whitespace difference or character substitution drops the entire crop score to 0.

Autoregressive vision-language decoding on Apple Silicon Metal GPU uses different matrix multiplication tiling, reduction order, and floating-point accumulation compared to NVIDIA CUDA kernels, causing tiny logit variations (~1e-4) that can flip argmax selections at near-tie probability boundaries.

Quickstart

1. Installation

pip install -r requirements.txt

2. Interactive Gradio Web Studio

Launch the visual interface with interactive bounding box hover tooltips, reading-ordered Markdown preview, and JSON inspection:

python app.py

Then open http://127.0.0.1:7860 in your browser.

3. End-to-End Python Pipeline

Run the full two-stage document pipeline (Stage 1 Layout Detection + Stage 2 Text Transcription):

from pipeline import IndicOCRPipeline

# Initialize end-to-end pipeline (auto-loads Stage 1 and Stage 2 on Apple Silicon)
ocr = IndicOCRPipeline()

# Process full document page
result = ocr.process("path/to/document.png")

# Reading-ordered Markdown
print(result.markdown)

# Access individual detected blocks
for block in result.blocks:
    print(f"#{block.order} [{block.label}] {block.bbox_xyxy}: {block.text}")

4. Single Crop CLI Transcription

Quickly transcribe a specific cropped text line or region:

python transcribe.py --image path/to/crop.png

5. Direct mlx_vlm Usage

Load the recognizer directly using standard MLX VLM:

from mlx_vlm import generate, load
from PIL import Image

model, processor = load("hari31416/indic-ocr-mlx-4bit")
prompt = processor.apply_chat_template(
    [
        {
            "role": "user",
            "content": [
                {"type": "image"},
                {"type": "text", "text": "Transcribe the text in this image."},
            ],
        }
    ],
    add_generation_prompt=True,
    tokenize=False,
)

image = Image.open("crop.png").convert("RGB")
output = generate(model, processor, prompt=prompt, image=[image], max_tokens=256)
print(output.text.strip())

Repository Structure

├── README.md                      # Model card and parity benchmarks
├── app.py                         # Interactive Gradio web application
├── pipeline.py                    # Importable end-to-end Python pipeline
├── transcribe.py                  # Single-crop CLI transcription script
├── config.json                    # Recognizer model configuration
├── generation_config.json         # Generation hyperparameter defaults
├── model.safetensors              # 4-bit recognizer weights (~636 MB)
├── model.safetensors.index.json   # Weight index mapping
├── processor_config.json          # Vision & image processor configuration
├── tokenizer.json                 # Sarvam-30B tokenizer
├── tokenizer_config.json
├── chat_template.jinja            # Multimodal chat template
├── requirements.txt               # Minimal dependencies
├── .gitattributes                 # Git LFS tracking configuration
└── layout/                        # Legacy experimental tensor export, not used at runtime
    ├── config.json
    └── model.safetensors          # PP-DocLayoutV3 tensors; requires an MLX model implementation

License and Citations

This model is distributed under the Indic Open Model License v1.0. See the Indic Open Model License and the Plain-Language License Deed for complete terms.

If using this export, please cite the original Bodhan AI project:

@misc{bodhan2026indicocr,
  author = {Bodhan AI},
  title = {IndicOCR: High-Performance Document OCR for Indic Languages},
  year = {2026},
  url = {https://huggingface.co/bodhan-ai/indic-ocr}
}