Indic-OCR (INT8 ONNX & GGUF)

Edge-Optimized, Quantized Document Parsing & Multilingual OCR for English and 22 Scheduled Indian Languages.

This repository provides high-performance, quantized INT8 ONNX and GGUF derivatives of Bodhan AI's Indic-OCR model. Designed specifically for low-latency, privacy-preserving, on-device document extraction, it allows offline execution on commodity CPUs, edge devices, and consumer hardware without requiring high-end server GPUs.

Mandatory Attribution: Built with Indic-OCR from Bodhan AI / AI4Bharat.

Model Provenance & Heritage

• Derivative Author: Community Contributor (2026)
• Direct Upstream Model: bodhan-ai/indic-ocr
• Original Authors: IITM BODHAN-AI FOUNDATION & AI4Bharat, supported by the Ministry of Education, Govt. of India.
• Upstream License: Indic Open Model License v1.0 (Plain language deed: LICENSE_DEED.md)
• Base Neural Architectures:
  • Layout Engine: RT-DETR (~33M params, Apache 2.0)
  • Recognition Engine: Qwen2.5-VL / Qwen3.5 vision-language architecture (~0.8B params, Apache 2.0)

Pipeline Architecture

Indic-OCR operates as an integrated two-stage cooperative pipeline:

Document Image (Scan / Photo / PDF)
                │
                ▼
  ┌───────────────────────────┐
  │  Stage 1: IndicDocLayout  │  RT-DETR (33M params)
  │  (Layout & Reading Order) │  Outputs: Text blocks, tables, titles, figures
  └─────────────┬─────────────┘
                │
                ▼
  ┌───────────────────────────┐
  │  Stage 2: IndicBlockOCR   │  Qwen VLM (0.8B params)
  │  (Block-Level Recognition)│  Transcribes text with natural reading order
  └─────────────┬─────────────┘
                │
                ▼
  Structured Document Output (JSON / Markdown / Plain Text)

1. Stage 1 — IndicDocLayout: Detects semantic bounding boxes and categorizes regions into 10 document classes: text, title, list, table, figure, header, footer, caption, formula, and stamp. It simultaneously establishes natural top-to-bottom, column-aware reading order.
2. Stage 2 — IndicBlockOCR: Crops each block and performs layout-aware transcription preserving intra-block line breaks, tabular formatting, and complex Indic conjuncts (ଯୁକ୍ତାକ୍ଷର, संयुक्ताक्षर, etc.).

Quantization Benchmarks

All models were evaluated on Intel/AMD x86_64 CPU cores (single-socket, AVX2 enabled):

1. Model Storage Footprint

2. Runtime Latency (CPU)

Verified on formal Odia broadcast news document segments: Produced 100% exact character reproduction (କେଉଁଠି ପାଣିରେ ବୁଡ଼ିଲା ଘରଦ୍ୱାର ତ କେଉଁଠି ରାସ୍ତାରେ ଛୁଟୁଛି ବନ୍ୟା ଜଳ...).

Repository Contents

├── .gitattributes
├── LICENSE                    # Bodhan AI Open Model License v1.0
├── LICENSE_DEED.md            # Plain language license summary
├── NOTICE.md                  # Attribution and architectural credits
├── README.md                  # This documentation
├── ARCHITECTURE.md            # In-depth architectural specification
├── TROUBLESHOOTING.md         # Common execution tips
├── indic-ocr-banner.png       # Visual banner
├── requirements.txt           # Python dependencies
├── indic_ocr.py               # Complete Python pipeline wrapper
├── idp_*.py                   # Modular inference engine components
├── config/
│   ├── config.json            # Pipeline routing specification
│   ├── layout_config.json     # IndicDocLayout configuration
│   ├── ocr_config.json        # IndicBlockOCR configuration
│   ├── generation_config.json # Autoregressive generation parameters
│   ├── processor_config.json  # Image preprocessor parameters
│   ├── tokenizer_config.json  # Tokenizer settings
│   └── chat_template.jinja    # Jinja conversation formatting template
├── tokenizer/
│   └── tokenizer.json         # Vocabulary and merges (33.6 MB)
├── onnx/
│   ├── indic_doclayout_int8.onnx          # Layout detector (INT8, 107.3 MB)
│   ├── indic_doclayout.onnx               # Layout detector (FP32, 127.5 MB)
│   ├── indic_block_ocr_vision_int8.onnx   # Vision encoder (INT8, 93.2 MB)
│   └── indic_block_ocr_vision.onnx        # Vision encoder (FP32, 336.0 MB)
├── gguf/
│   ├── indic-ocr-q8_0.gguf                # Block OCR (Q8_0, 792.5 MB)
│   ├── indic-ocr-q5_k_m.gguf              # Block OCR (Q5_K_M, 566.2 MB)
│   ├── indic_doclayout-q8_0.gguf          # Layout detector (Q8_0, 57.1 MB)
│   └── indic_doclayout-f16.gguf           # Layout detector (F16, 63.9 MB)
└── metadata/
    ├── manifest.json          # Deployment metadata & engine signatures
    └── checksums.sha256       # Cryptographic verification hashes

Supported Languages (23 Languages)

Quickstart: Python Inference

1. Installation

pip install onnxruntime opencv-python pillow numpy torchvision

2. Running ONNX INT8 Layout + Vision Pipeline

import cv2
import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download

REPO_ID = "adidsh/indic-ocr-int8-onnx"

# 1. Download INT8 ONNX models
layout_model_path = hf_hub_download(repo_id=REPO_ID, filename="onnx/indic_doclayout_int8.onnx")
vision_model_path = hf_hub_download(repo_id=REPO_ID, filename="onnx/indic_block_ocr_vision_int8.onnx")

# 2. Initialize ONNX Runtime session
sess_opts = ort.SessionOptions()
sess_opts.intra_op_num_threads = 4
layout_session = ort.InferenceSession(layout_model_path, sess_opts, providers=["CPUExecutionProvider"])
vision_session = ort.InferenceSession(vision_model_path, sess_opts, providers=["CPUExecutionProvider"])

print("Indic-OCR INT8 ONNX Pipeline Initialized Successfully!")

3. Running with GGUF via llama.cpp

# Run text block generation with 8-bit quantized GGUF
./llama-cli \
  -m indic-ocr-q8_0.gguf \
  --prompt "<|image_pad|> Transcribe the document block accurately:" \
  -n 512 \
  --threads 4

Licensing & Attribution

This model is licensed under the Indic Open Model License v1.0.

Required Attribution

Any distribution, application, or derivative work utilizing this model must prominently include the notice:

"Built with Indic-OCR from Bodhan AI / AI4Bharat."

Key License Provisions

1. Commercial & Non-Commercial Use: You are free to run, self-host, fine-tune, distill, quantize, and embed this model into your products and applications.
2. Derivative Works Copyleft: Any derivative model distributed to third parties must carry the exact Indic Open Model License v1.0.
3. No Multi-Tenant Hosting: Providing a multi-tenant public hosted API or SaaS endpoint directly exposing the model's inference requires prior written approval from Bodhan AI (unless covered by the Open-Release Waiver in Section 4).
4. Acceptable Use: Strict prohibition against unlawful activities, non-consensual biometric processing, mass surveillance, and disinformation campaigns.
5. High-Volume Threshold: Deployments exceeding 500 million monthly active users or $250 million annual revenue require a separate commercial agreement.

For full legal terms, see LICENSE and LICENSE_DEED.md.