DE·HTR v2 — German handwriting recognition in the browser

Offline German handwritten text recognition (HTR / Handschrifterkennung) that runs entirely client-side: a 10.4 MB int8 ONNX model for ONNX Runtime Web, no image upload, no per-call cost, no GPU required.

de-htr-web-v2 is a compact line-level CTC recognizer for modern German handwriting (Kurrent/Sütterlin are out of scope). It is the successor to naeyn/de-htr-web v1 and beats it on the identical sealed benchmark while carrying a fully audited, license-clean training-data ancestry.

• Live demo: https://de-htr-web.vercel.app
• Input: one pre-cropped handwriting line image (any common format)
• Output: Unicode transcription via greedy CTC decoding
• Privacy: after the page and model load, inference makes no network requests

Results

Benchmark: 601 handwritten lines from 78 held-out writers of the ScaDS.AI German handwriting dataset — writer-disjoint from all training data, aligned with v1's sealed test writers (38 shared + 40 additional unseen). Greedy CTC decode, no language model, batch size 1 (exactly how the browser runs it). v1 was re-evaluated under the identical protocol.

Per-writer error is heavy-tailed (median writer ≈ 6–8% CER, hardest writers 30%+): expect noticeably better results on clean, well-scanned handwriting and worse on faint pencil, dense corrections, or unusual styles.

Release identity

The companion browser app ships this model as of source tag v0.2.0 of naeyn/de-htr-web.

Architecture: ImageNet-initialized **ResNet18 stem (through layer3) + 3×BiLSTM-256

• CTC head**, 10.06M parameters. Grayscale input at height 64, dynamic width up to 1024, output stride 4, 112-symbol alphabet (blank + 111 characters covering ≥ 99.99% of benchmark characters).

⚠️ Breaking changes vs v1

v2 is not weight- or interface-compatible with v1 — that is why it lives in a separate repository. Read config.json (format_version: 2) instead of hard-coding:

Usage

JavaScript (ONNX Runtime Web — the intended path)

import * as ort from "onnxruntime-web";

const config = await (await fetch("config.json")).json();
const session = await ort.InferenceSession.create("model.onnx");

// preprocess: grayscale, resize to height 64 keeping aspect ratio,
// right-pad width to a multiple of 4 with white, scale to [0,1]
const tensor = new ort.Tensor("float32", pixels, [1, 1, 64, width]);
const { logits } = await session.run({ image: tensor });

// greedy CTC decode: per-frame argmax, collapse repeats, drop blank (index 0)

Python (onnxruntime)

import numpy as np, onnxruntime as ort
from PIL import Image

cfg = json.load(open("config.json"))
img = Image.open("line.png").convert("L")
w = max(4, round(img.width * 64 / img.height) // 4 * 4)
img = img.resize((w, 64))
x = np.asarray(img, dtype=np.float32)[None, None] / 255.0

session = ort.InferenceSession("model.onnx")
logits = session.run(None, {"image": x})[0]          # (1, w//4, 112)
ids = logits.argmax(-1)[0]                            # greedy CTC
text = "".join(cfg["alphabet"][i - 1] for i, prev in zip(ids, np.r_[0, ids[:-1]])
               if i != 0 and i != prev)
print(text)

Training recipe

Trained from scratch (no v1 weights) on a single RTX 5070 Ti in ~3 hours:

Ablation headlines (full table in the research repo): ImageNet stem −1.3 CER; adding the fhswf corpus −0.94; dropping synthetic-data batch mixing −1.28 at matched exposure; synthetic-only pretraining saturates at 0.25 epoch-equivalents and then hurts transfer; frame-wise two-seed ensembling fails for CTC (alignment blur). The negative synthetic results are reported deliberately — the license-clean synthetic renderer and corpus lock ship as a separate research artifact.

Training data and provenance

The released weights were trained exclusively on real handwriting:

Not in the training path of these weights: synthetic rendered text, Leipzig corpora, VLM pseudo-labels, IAM. Splits are writer-disjoint and pinned; the sealed test writers were never trained on. Source images are not redistributed here. Full dataset-level attribution: NOTICE-training-data.md.

Limitations

• Modern German handwriting only — no historical scripts (Kurrent, Sütterlin), no printed-text OCR guarantee, weak on heavy math/symbols.
• Line-level input: you must segment pages into lines first (the demo app crops lines; full-page layout analysis is out of scope for the model).
• Heavy-tailed writer difficulty (see Results) — evaluate on your own handwriting distribution before depending on it.
• Alphabet is 112 symbols; characters outside it (e.g. curly quotes, accented Romance characters) cannot be emitted.
• Greedy decode has no language model; a domain lexicon or LM rescoring layer can further reduce word errors downstream.

License and attribution

Weights: Apache-2.0 for the rights held by the project. ScaDS.AI data used under CC BY 4.0 (Burghardt, Alzin, Nestler et al., ScaDS.AI / Universität Leipzig; changes: cropping, grayscale normalization, NFC text normalization, writer-disjoint re-splitting). fhswf data used under AFL-3.0 for training only. Dataset-level attribution in NOTICE-training-data.md.

Citation

@software{de_htr_web_v2,
  title = {DE·HTR v2: German handwriting recognition in the browser},
  author = {naeyn},
  year = {2026},
  version = {2.0.0},
  url = {https://huggingface.co/naeyn/de-htr-web-v2},
  license = {Apache-2.0}
}