Surya OCR 2 — Persian Handwriting 369K
This is a full-parameter fine-tune of
datalab-to/surya-ocr-2 on all
369,000 pages of
Reza2kn/persian-handwriting-pages-369k.
Training
- Epochs: 1
- Rows: 369,000
- Optimizer: AdamW
- Precision: BF16
- Learning rate: 1e-05
- Per-GPU batch: 8
- Global batch: 64
- GPUs: 8 x NVIDIA RTX PRO 6000 Blackwell Server Edition
- Update steps: 5,766
- Parameter strategy: full-parameter fine-tuning (not LoRA)
- Prompt:
OCR this Persian document image. Return only the complete transcription in reading order.
Fixed 69-page Persian before/after diagnostic
Both rows use the same model family, BF16 inference, prompt, first 69 dataset
pages, normalization, and word-error implementation.
The exact ordered image/label set is fingerprinted as
09ee91df3d0a920a14233b9ad769fe29a5861384a2fe86b409c9979155682e19 in both summaries.
These 69 pages are part of the 369,000-page training corpus. This is therefore an in-training-set acquisition diagnostic, useful for measuring the requested before/after fit but not a held-out estimate of Persian generalization.
| Model | WER | Word errors / reference words | Exact pages |
|---|---|---|---|
| Original Surya OCR 2 | 149.28% | 18,797 / 12,592 | 0/69 |
| This checkpoint | 2.89% | 364 / 12,592 | 46/69 |
WER can exceed 100% because insertions count as errors. The checkpoint changes WER by -146.39 percentage points (-98.06% relative) on this fixed diagnostic.
Held-out Persian retention
These checks were added after training to measure whether the full-parameter
fine-tune preserved Persian OCR outside the composed 369K training pages. Both
models used identical images, prompt, normalization, decoding, and BF16
inference. Raw predictions and summaries are under evaluation/.
Printed Persian: ParsynthOCR test split
The first 69 examples from the held-out test split of
mainkilora/parsynth-ocr-200k
were evaluated at dataset revision
b8f6db1bb635a801f8f222144643d2c52db49b18. The exact ordered image/text set
has fingerprint 677c9ff212f5a2e4bea545f235e829fcf1231373e5c23404f2f87b6045ca3753.
| Model | WER | Word errors / reference words | Exact images |
|---|---|---|---|
| Original Surya OCR 2 | 23.87% | 37 / 155 | 46/69 |
| This checkpoint | 168.39% | 261 / 155 | 21/69 |
This checkpoint substantially degrades on this printed-Persian word-image sample and frequently emits extra text.
Real Persian handwriting: PHTD holdout
This check uses 65 real Persian handwritten word crops from the separately
prepared PHTD real-holdout set. Its labeled manifest fingerprint is
f225f23fbec246e1ff034693647ef6f57ed95899824b056993b99bcfcf7a92ab.
Because every reference contains one word, exact recognition and CER are more
informative than WER when a model emits multiple words.
| Model | WER | CER | Exact words | Multiword outputs |
|---|---|---|---|---|
| Original Surya OCR 2 | 95.38% | 54.01% | 8/65 | 5/65 |
| This checkpoint | 116.92% | 74.56% | 18/65 | 16/65 |
Exact recognition more than doubles, but aggregate WER and CER worsen because the checkpoint produces more multiword hallucinations. This is not clean retention: it improves some handwritten words while becoming less reliable overall.
Narrow retention canary
This 69-item English handwritten-word crop test is a deliberately narrow forgetting canary, not a comprehensive general OCR benchmark. Surya sometimes emits long layout-style text for these tiny crops, so exact matches are more interpretable than its inflated WER.
| Model | WER | Exact words |
|---|---|---|
| Original Surya OCR 2 | 800.00% | 24/69 |
| Step 500 | 832.20% | 9/69 |
| Step 2,000 | 988.14% | 6/69 |
| Step 3,000 | 805.08% | 6/69 |
| Final checkpoint | 1266.10% | 5/69 |
Raw predictions, references, summaries, and manifests are included under
evaluation/ for auditing.
Limitations
The training data is synthetic composed Persian handwriting on photographed paper backgrounds. This checkpoint may trade some of the base model's broader OCR/layout behavior for specialization. Evaluate it on your own printed, handwritten, multilingual, and layout-heavy documents before deployment.