TrOCR Spanish Handwritten
This model is a fine-tuned version of microsoft/trocr-base-handwritten specifically adapted for modern Spanish handwriting recognition (HTR).
Standard TrOCR models are trained primarily on English datasets (such as IAM), which causes severe hallucinations and vocabulary mismatches when processing Spanish text. This checkpoint adapts the language decoder to Spanish orthography, diacritics (á, é, í, ó, ú, ü), and special characters (ñ, ¿, ¡).
Model Details
- Architecture: vision Transformer (ViT) encoder + RoBERTa causal language decoder.
- Base Checkpoint:
microsoft/trocr-base-handwritten - Language: Spanish (
es) - Task: text Line Optical Character Recognition / Transcription
- Input: cropped line images of handwritten text.
Training Data & Methodology
- Synthetic Spanish Corpus: trained on 3,000 synthetic handwriting line crops generated across 41 varied cursive, printed, and script handwriting TrueType fonts.
- Visual Augmentations: includes paper backgrounds (lined notebooks, grids, blank textures), realistic ink color variations (ballpoint blue, black, sepia), slight perspective distortion, and optical blur.
- Optimization: encoder weights were frozen to retain base stroke detection while the autoregressive decoder was adapted to modern Spanish vocabulary, poetry excerpts, names, and date structures.
Comparison: Base Model vs. This Checkpoint
Base microsoft/trocr-base-handwritten | ifesther/trocr-spanish-handwritten (This Model) |
|---|---|
(( N &e cìndas, qe la vìda es 0511... (English hallucination) | No te rindas, que la vida es eso, |
Aarie Bmeclett; | - Mario Benedetti. |
Limitations
- Line-Level only: trOCR is designed to read single lines of text. Full pages must first be segmented into individual line crops using line detection heuristics or specialized detectors.
- Archaic calligraphy: optimized for contemporary handwriting; performance on historical manuscripts may vary.