PaddleOCR-VL-1.6 Bangla

A Bangla fine-tune of PaddleOCR-VL-1.6 for printed Bangla and English documents.

Highlights

• Accurate printed Bangla: under 1% character error on held-out printed words, against 88% for the base model.
• Full-page reading with boxes: every line of a page with its bounding box, at 0.15% character error on held-out Bangla pages.
• Drop-in replacement: the same prompts and output format as the base model, released as merged weights, a 24 MB LoRA adapter, and GGUF for llama.cpp.

Model details

Evaluation

Character error rate (CER) on held-out test sets never used in training; lower is better. Each line is read independently with greedy decoding.

— not measured on the base model. ¹ The base model repeats text on many handwritten lines, so its error rate exceeds 100%.

Document pipeline. Spotting each page and then re-reading every detected row with OCR:, on 18 synthetic pages (12 Bangla, 6 English): 99.3% of words correct and all 728 amounts and dates found, against 55.1% of words for the base model alone.

Per-set results are in eval/scores.json. CER is the edit distance divided by the reference length, after Unicode NFC and whitespace normalisation.

Intended use

Text extraction from printed Bangla and English documents, including scans: statements, invoices, forms, letters and books.

Limitations

• Handwriting: about 31% character error on held-out writers; not recommended for handwritten documents.
• Isolated words from real-world documents: about 10% character error.
• Evaluation scope: measured on public and synthetic test sets; not yet on a hand-labelled set of real business documents.
• As with any OCR system, verify extracted figures before relying on them.

Usage

The model keeps the base model's two prompts:

• OCR: with a cropped line or region returns its text.
• Spotting: with a full page returns every line as its text followed by eight location tokens: the four corners of its box, clockwise from the top left, on a 0–1000 scale. For example: 01/07/2025 প্রারম্ভিক স্থিতি 21,11,701.00<|LOC_77|><|LOC_119|><|LOC_337|><|LOC_120|><|LOC_337|><|LOC_133|><|LOC_77|><|LOC_131|>

For best accuracy, spot the page (scaled to at most about 1.6 million pixels), then re-read each detected row with OCR:, cropped from the full-resolution page.

Transformers

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

MODEL = "hasanmdsadadul/PaddleOCR-VL-1.6-Bangla"
device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(MODEL)
model = AutoModelForImageTextToText.from_pretrained(MODEL, dtype=torch.bfloat16).to(device).eval()

def ask(image, task, max_new_tokens=4096):
    messages = [{"role": "user", "content": [{"type": "image", "image": image}, {"type": "text", "text": task}]}]
    inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True,
                                           return_tensors="pt").to(device)
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False, use_cache=True)
    return processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False)

print(ask(Image.open("line.png").convert("RGB"), "OCR:", 256))
print(ask(Image.open("page.jpg").convert("RGB"), "Spotting:"))

PEFT adapter

from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained("PaddlePaddle/PaddleOCR-VL-1.6", dtype=torch.bfloat16).to(device)
model = PeftModel.from_pretrained(base, MODEL, subfolder="adapter").eval()

llama.cpp

The gguf/ folder contains the merged model (BF16), the adapter on its own, and the base model's vision projector with clip.vision.image_max_pixels set to 1,605,632, as the base model recommends for spotting. Use --special to keep the location tokens in the output. Tested with llama.cpp b10809.

llama-server -m gguf/PaddleOCR-VL-1.6-Bangla.gguf --mmproj gguf/mmproj-spotting.gguf --temp 0 --special

Alternatively, apply the adapter to the base model's GGUF from PaddlePaddle/PaddleOCR-VL-1.6-GGUF:

llama-server -m PaddleOCR-VL-1.6-GGUF.gguf --mmproj gguf/mmproj-spotting.gguf \
  --lora gguf/PaddleOCR-VL-1.6-Bangla-lora.gguf --temp 0 --special

Then send an image with Spotting: or OCR: as the text through the chat completions API.

Training data

Only public datasets and synthetic documents were used; no private documents.

Sources marked "not stated" publish no license. They were used in good faith, and their owners may request removal.

License

The model weights are released under CC BY-SA 4.0, as required by the share-alike license of the Wikisource text used in training. The base model, PaddleOCR-VL-1.6, is licensed under Apache 2.0 by PaddlePaddle; its license is included as LICENSE-PaddleOCR-VL-1.6.

Acknowledgements

PaddlePaddle for PaddleOCR-VL-1.6, and the creators of the datasets listed above.