trocr-large-uk-handwritten-real

A TrOCR-Large (VisionEncoderDecoderModel, ~558M params) recognizer for line-level handwritten Ukrainian text.

This checkpoint continues the mixed (real + synthetic) pretrain Hukyl/trocr-large-uk-handwritten on real handwriting only, to wash out synthetic-distribution artefacts. Like the pretrain, it is meant as a starting point for downstream HTR fine-tunes.

TL;DR

Intended use

Recognizing single cropped lines of handwritten Ukrainian text, downstream of a line/region detector or on pre-segmented lines. Useful directly as a general handwriting reader, or as an init for fine-tuning on a target domain.

How to use

Note: load the processor from this repo, not from microsoft/trocr-large-handwritten — the tokenizer here is an extended Cyrillic vocabulary (50336 vs stock 50265).

from PIL import Image
from transformers import VisionEncoderDecoderModel, TrOCRProcessor

repo = "Hukyl/trocr-large-uk-handwritten-real"
processor = TrOCRProcessor.from_pretrained(repo)          # load from THIS repo
model = VisionEncoderDecoderModel.from_pretrained(repo).eval()

crop = Image.open("line_crop.png").convert("RGB")          # one cropped text line
pixel_values = processor(images=crop, return_tensors="pt").pixel_values
generated = model.generate(pixel_values, max_new_tokens=256, num_beams=1)  # greedy
text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)

Lineage & training curriculum

Full fine-tune, optimizer: AdamW via the 🤗 Seq2SeqTrainer, cosine LR decay, max_target_length 256, online augmentation, seed 42, single L40S.

1. Base — Kansallisarkisto/cyrillic-htr-model (Apache-2.0): microsoft/trocr-large-handwritten fine-tuned on ~30k historical Cyrillic rows by the Finnish National Archives; extended tokenizer (vocab 50336).
2. Mixed pretrain — Hukyl/trocr-large-uk-handwritten (6 ep, LR 4e-5): UkrHandwritten ×3 + Cyrillic Handwriting + part of pumb-ai/synthetic-cyrillic-large.
3. Real-only consolidation (16 ep, LR 2e-5, warmup 0.05): UkrHandwritten (oversampled ×2) + the full Cyrillic Handwriting Dataset, ≈145k samples per epoch, no synthetic data. The published weights are the best-val-CER epoch (16 of 16).

Hyperparameters (as launched)

Training ran 72,656 steps in ~10.4h.

Online data augmentation

Online augmentation was applied during training. Only training crops were augmented (no augmented validation was measured).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. All training crops are handwritten lines, so a single handwriting-oriented transform set was used.

These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting deformation, widening the range of appearances beyond the raw training crops.

Results

In-domain held-out validation

Held-out 1.8% of the real training corpora (UkrHandwritten + Cyrillic Handwriting, same distribution as training). Greedy decode.

Out-of-domain transfer probe (zero-shot)

The same checkpoint evaluated zero-shot on a held-out page-level split of the Rukopys dataset (handwritten Ukrainian document pages) — a distribution the model never trained on. 1000-sample draw, seed 42, greedy decode, max_new_tokens=512.

The gap between in-domain (0.0249) and zero-shot documents (0.1999) is the expected domain shift from clean line corpora to document crops; a fine-tune on in-domain document data closes most of it. We also acknowledge that printed/table/annotation n is quite small, so measuring CER against them is quite noisy.

Limitations & biases

• General, not specialized: expect higher error on noisy archival scans until fine-tuned (see the zero-shot transfer probe).
• Outputs may differ from the source in casing/punctuation — they are model transcriptions, not faithful transcriptions.
• Single seed and validation split — no across-run variance estimate.

Training data & attribution

Attribution is mandatory for the UkrHandwritten dataset (CC BY-SA 4.0) — please keep this credit if you redistribute or build on this model.