Chitrapathak-2

Overview

Chitrapathak-2 (Chitra: Image; Pathak: Reader) is a VLM-based multilingual Optical Character Recognition (OCR) system designed specifically for the linguistic diversity and document complexity of the Indian ecosystem. Trained for high-fidelity OCR on Indic language book pages, Chitrapathak-2 demonstrates strong generalization across 10 major Indian languages and English. It is the second model in the Chitrapathak OCR series, continuing the effort to build robust OCR systems for Indic scripts and multilingual documents.

Model Summary

Usage

Using transformers

from PIL import Image
from transformers import AutoTokenizer, AutoProcessor, AutoModelForImageTextToText

model_path = "krutrim-ai-labs/Chitrapathak-2"

model = AutoModelForImageTextToText.from_pretrained(
    model_path, 
    torch_dtype="auto", 
    device_map="auto", 
    attn_implementation="flash_attention_2"
)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(model_path)
processor = AutoProcessor.from_pretrained(model_path)


def perform_ocr(image_path, model, processor, max_new_tokens=4096):
    image = Image.open(image_path)
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": [
            {"type": "image", "image": f"file://{image_path}"},
            {"type": "text", "text": "Perform OCR on this image and transcribe all visible text exactly as it appears."},
        ]},
    ]
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = processor(text=[text], images=[image], padding=True, return_tensors="pt")
    inputs = inputs.to(model.device)
    
    output_ids = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)
    generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
    
    output_text = processor.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)
    return output_text[0]

image_path = "/path/to/your/document.jpg"
result = perform_ocr(image_path, model, processor, max_new_tokens=15000)
print(result)

Using vLLM

1. Start the vLLM server.

vllm serve krutrim-ai-labs/Chitrapathak-2

2. Predict with the model

from openai import OpenAI
import base64

client = OpenAI(api_key="123", base_url="http://localhost:8000/v1")

model = "krutrim-ai-labs/Chitrapathak-2"

def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")

def perform_ocr(img_base64):
    response = client.chat.completions.create(
        model=model,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image_url",
                        "image_url": {"url": f"data:image/png;base64,{img_base64}"},
                    },
                    {
                        "type": "text",
                        "text": "Perform OCR on this image and transcribe all visible text exactly as it appears.",
                    },
                ],
            }
        ],
        temperature=0.0,
        max_tokens=15000
    )
    return response.choices[0].message.content

test_img_path = "/path/to/your/document.jpg"
img_base64 = encode_image(test_img_path)
print(perform_ocr(img_base64))

Evaluation Results

Indic OCR Performance

• The above table shows the performance comparison of models on the IndicVisionBench-OCR benchmark.
• The metrics used above are word-level ANLS and character-level ANLS.
• The best value in each column is in bold and the second best is underlined.
• Chitrapathak-2 delivers SOTA OCR accuracy in Telugu and is remarkably close to Gemini-2.5 in other Indic languages, with an average difference of only 2.21 (word) and 1.83 (char) across the nine Indic languages, and a maximum gap of just 4.83 (word) and 4.10 (char).

English OCR Performance

• The above table shows the performance comparison of models on Synthdog and SROIE benchmarks.
• The metrics for Synthdog are ANLS-word and ANLS-char. In these metrics, lower is better.
• The metric for SROIE is %Match, indicating the percentage of fields that matched exactly with the ground truth. Here, higher is better.
• As we see, Chitrapathak-2 retains much of the English OCR capabilities of its base model Nanonets-OCR2-3B.

We also evaluated the model on the Old books OCR dataset, which consists of scanned book pages in English.

Token Efficiency and Latency Breakdown of Chitrapathak-2

Note that the above latency values are calculated assuming the size of the input image to be ~1024x1024.

Observations

• TTFT (Time-to-First-Token): ~125 ms
• Inter-token latency: ~4 ms per token
• Language impact: Latency varies with tokenization efficiency.
  • English and Hindi → Lower latency due to compact token-to-word ratios.
  • Telugu and Malayalam → Higher latency due to fragmented tokenization (larger number of tokens per word).

Example OCR Outputs

Highlights

• Supports 9 major Indic languages + English
• Optimized for printed book pages, PDF documents, and scanned text
• Strong layout robustness — handles multi-column, multi-font, and dense paragraphs with ease
• Second-best overall model on IndicVisionBench-OCR, ranking just behind Gemini-2.5, and the best-performing model for Telugu
• Compatible with vLLM and Hugging Face inference pipelines
• Multilingual generalization across diverse font styles, layouts, and page qualities

Limitations

• The model is only capable of OCR and is not meant for a document intelligence use-case.
• Given an image, the model only returns the OCR transcription in its default output format.
• Performance may drop on handwritten, noisy, or low-resolution images.
• Some degradation may be observed for rare Indic/English scripts or non-book domains (e.g. forms).
• Performance degradation is observed on Index-page layouts and other complicated page layouts.

License

This model is distributed under the Krutrim Community License Agreement v1.0.
Ensure compliance before any commercial or redistributed usage.

Update: This model is released only for research purposes since the underlying Qwen-2.5-VL model has changed from Apache to a more restrictive license.

Citation

If you use Chitrapathak-2 in your research, please cite:

@misc{faraz2026indicocr,
      title={Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems}, 
      author={Ali Faraz and Raja Kolla and Ashish Kulkarni and Shubham Agarwal},
      year={2026},
      eprint={2602.16430},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2602.16430}, 
}

Acknowledgements

Chitrapathak-2 builds upon the foundations of the following projects and open-source efforts:

• Nanonets-OCR2-3B
• Transformers
• TRL (Transformer Reinforcement Learning)
• PyTorch