Azerbaijani line recognizer (CRNN + CTC)

Recognises a single cropped line of Azerbaijani text, in Latin or Cyrillic script. 8.5M parameters, 34 MB — it loads in under a second and runs on CPU, which is the point: general vision-language models read this material well but cost seconds per line and gigabytes of memory.

This is a line recogniser, not a page OCR system. Pair it with a text detector (PP-OCRv5 detection works) to process whole pages.

Results on LocalDoc/azerbaijani-ocr-benchmark

Diacritics CER covers ə ı ğ ş ç ö ü İ and the Cyrillic ә ҝ ҹ ғ ј һ ө ү. For this language it matters more than the overall figure: those letters are a minority of characters and carry most of the meaning that gets lost.

Usage

import torch
from PIL import Image
from huggingface_hub import hf_hub_download
from modeling import load, preprocess, ctc_greedy

path = hf_hub_download("LocalDoc/azerbaijani-ocr-crnn", "model.pt")
model, itos = load(path)

img = Image.open("line.png")     # crop of ONE text line
with torch.no_grad():
    logits = model(preprocess(img))
print(ctc_greedy(logits, itos))

Input is normalised to 48 px height, any width. The backbone divides width by 4, so a 400 px crop yields 100 CTC time steps.

Whole pages and PDFs

pipeline.py in this repo wires the recogniser to a text detector. Detection is language-independent — a general model handles it, and only recognition needs to know Azerbaijani.

pip install paddleocr paddlepaddle pymupdf

from PIL import Image
from pipeline import PageOCR

ocr = PageOCR()

print(ocr.page(Image.open("scan.png")))

for i, text in enumerate(ocr.pdf("book.pdf", pages=range(0, 5),
                                 join_hyphens=True)):
    print(f"--- page {i} ---\n{text}\n")

The pipeline detects line boxes with PP-OCRv5_server_det, pads them vertically without letting a box run into its neighbour, sorts them into reading order, drops vertical text, and recognises in width-sorted batches. join_hyphens=True reattaches words split across lines.

Two settings worth knowing. UNCLIP controls how much the detector inflates each box: raise it if descenders on ç ş ğ come out clipped, lower it if crops start capturing the line above. dpi=300 is a floor — below roughly 150 dpi on the physical page, line height drops under 20 px and the diacritics stop existing as pixels rather than being merely hard to read.

Multi-column pages are not handled: reading order assumes one column, so lines from adjacent columns interleave. Split the page at the gutter first.

Honest limitations

Training labels were not human-verified. They come from OCR text layers already present in the source PDFs, so the model inherited that engine's systematic errors. The single most common residual error is predicting a where the print shows ə — the model learned that from its labels, not from the pixels. Cleaning the labels, rather than changing the architecture, is the next real improvement.

The Cyrillic figure is optimistic. Cyrillic is a small minority of the training corpus, and the Cyrillic side of the benchmark comes from few books. Treat 0.0044 as evidence the model can read those books, not as a general Cyrillic result.

Source resolution sets a floor. Text lines in these scans run about 20 px high. At that scale и/н/п/м differ only in the count of vertical strokes and are genuinely ambiguous — both the reference labels and the model guess. No architecture change reaches past that.

Greedy CTC decoding. No language model. A character n-gram model in a prefix beam search typically cuts CER substantially on low-resource languages and requires no retraining.