Model description

Model Name: latvian-large-handwritten

Model Version: latvian-large-june2026-400-5e05

Model Type: Transformer-based encoder-decoder for OCR

Base Model: microsoft/trocr-large-handwritten

Purpose: Handwritten text recognition

Languages: Latvian

License: Apache 2.0

This is a fine-tuned model for Latvian handwriting recognition, trained with computing resources generously provided by CSC – IT Center for Science on the Puhti and LUMI supercomputers.

This model was developed in the ArchXAI project funded by the Central Baltic Programme.

Model Architecture

The model is based on a Transformer architecture with an encoder-decoder setup, similar to TrOCR from Li et. al. (2023):

• The encoder processes an image of a single line of text into a sequence of hidden states.
• The decoder attends to the hidden states from the encoder using cross-attention, to generate the corresponding text output.

This model is a fine-tuned version of the original trocr-large-handwritten from Li et. al. (2023), for handwritten text recognition in primarily Latvian-language historical documents.

Intended Use

• Document digitization (e.g., archival work, historical manuscripts)
• Handwritten notes transcription

Training data

The training data consists of human-annotated samples of mainly handwritten text lines from historical documents in the collections of the National Archives of Latvia, with some printed and typed text lines included. The training set and validation set (but not the test set) were expanded with empty lines containing no text, from the collections of the National Archives of Finland.

Training set: 102 570 text lines (including 7106 empty lines)

Validation set: 13 076 text lines (including 888 empty lines)

Test set: 12 053 text lines

Evaluation

The following metrics were calculated on the test set (in-domain evaluation) using the evaluate library with default settings:

CER (character error rate): 0.0233

WER (word error rate): 0.1050

Used Hyperparameters

Train batch size per device: 8

Number of devices: 256

Learning rate: 5e-5

Scheduler: linear

Optimizer: AdamW

Number of epochs: 400

FP16 mixed precision training: False

Input image size: 192 x 1024

How to Use the Model

You can use the model for inference by loading the processor and model.

from transformers.models.vit.modeling_vit import ViTPatchEmbeddings, ViTEmbeddings
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

def load_custom_trocr_model():
    """Load a TrOCR model with custom image size support"""
    original_embeddings_forward = ViTEmbeddings.forward
    
    # Always apply patches for models saved with custom image sizes
    def universal_patch_forward(self, *args, **kwargs):
        pixel_values = args[0] if args else kwargs['pixel_values']
        embeddings = self.projection(pixel_values).flatten(2).transpose(1, 2)
        return embeddings
    
    def universal_embeddings_forward(self, *args, **kwargs):
        kwargs['interpolate_pos_encoding'] = True
        return original_embeddings_forward(self, *args, **kwargs)
    
    # Apply patches
    ViTPatchEmbeddings.forward = universal_patch_forward
    ViTEmbeddings.forward = universal_embeddings_forward
    
    # Load model and processor
    processor = TrOCRProcessor.from_pretrained("Kansallisarkisto/latvian-large-handwritten",
                                               use_fast=True,
                                               do_resize=True, 
                                               size={'height': 192,'width': 1024})
     
    model = VisionEncoderDecoderModel.from_pretrained("Kansallisarkisto/latvian-large-handwritten")
    
    return processor, model

# Load model and processor
processor, model = load_custom_trocr_model()

# Open an image of handwritten text
image = Image.open("path_to_image.jpg")

# Preprocess and predict
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(generated_text)

Limitations and Biases

The model was trained primarily on handwritten text that uses basic Latin characters and Latvian special characters. It has not been trained on non-Latin alphabets, such as Chinese characters or other writing systems like Arabic or Hebrew. The model may not generalize well to any other language than Latvian.

Future Work

Potential improvements for this model include:

• Expanding training data: incorporating more ground truth data
• Optimizing for specific domains: fine-tuning the model on domain-specific handwriting
• Pretraining: pre-training a fully Latvian-specific model instead of starting the fine-tuning from a model trained on English
• Out-of-domain generalization: studying how pre-training and fine-tuning could be optimized to maximize out-of-domain generalization of the fine-tuned model

Citation

If you use this model in your work, please cite it as:

@misc{latvian-large-handwritten,
  author = {Kansallisarkisto},
  title = {NAF Latvian HTR model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Kansallisarkisto/latvian-large-handwritten/}},
}

References

Li, M., Lv, T., Chen, J., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z. and Wei, F. 2023. TrOCR: Transformer-Based Optical Character Recognition with Pre-trained Models. Proceedings of the AAAI Conference on Artificial Intelligence. 37, 11 (Jun. 2023), 13094-13102. DOI:https://doi.org/10.1609/aaai.v37i11.26538.

Model Card Authors

Author: Kansallisarkisto

Contact Information: john.makela@kansallisarkisto.fi, ilkka.jokipii@kansallisarkisto.fi