Handwritten Meitei Mayek word recognition
The first segmentation-free recogniser of handwritten Meitei Mayek words: the stem and first three stages of ConvNeXt-T (started from a network trained on the TUMMHCD characters), a two-layer bidirectional LSTM and CTC over the 54 characters of everyday spelling, decoded by beam search with a character 6-gram language model (interpolated Kneser-Ney). Trained only on synthetic words composed from TUMMHCD training characters. In half of the training words, the signs that print places above a letter (ꯥ, ꯩ, ꯪ) are written beside it at the top, where handwriting often puts them, as with ꯦ; in the others they stand above it, as in print.
On 5,000 synthetic test words (TUMMHCD test characters) it reaches a character error rate of 0.33% and a word error rate of 2.04% with the language model (0.39% and 2.42% without). On the same words with every sign that print places above a letter written beside it at the top: character error rate 0.25%, word error rate 1.66% with the language model.
The earlier model (run round2_convnext_tummhcd_seed1, trained with every sign in its printed place), which the paper describes, is kept at revision paper-v1 of this repository.
| File | What |
|---|---|
recogniser.pt | the network (EMA weights) and its training settings; run round3_convnext_tummhcd, step 52000 |
char_lm.pkl | the character language model (mayek_htr.lm.CharLM) |
decoding.json | the language model's weight and the bonus per character, chosen on the synthetic validation set |
results.json | the scores above, as written by the evaluation |
web/model.onnx, web/lm.bin.gz | the same, for the browser demo (weights stored as float16) |
Use
git clone https://github.com/chingkheinganba231005/meitei-mayek-word-recognition
cd meitei-mayek-word-recognition && pip install -r requirements.txt -r requirements-htr.txt
huggingface-cli download Chingkheinganba/handwritten-meitei-mayek-word-recognition --local-dir word_model
from mayek_htr.reader import Reader
reader = Reader("word_model")
print(reader.read("word.jpg")) # one word, dark ink on light paper
print(reader.read("word.jpg", use_lm=False)) # the network alone (greedy decoding)
The image is cut to its ink and prepared as in the evaluation; the language model's weight
comes from decoding.json.
Words are written in everyday spelling (ꯏ for every i); mayek_words.charset.standard_spelling
renders the standard spelling (ꯢ after ꯥ, ꯣ, ꯨ).
Limitations
Real handwriting has been measured on one writer on a tablet only; other hands, pens and photographs will be read less reliably. Characters outside TUMMHCD (lum iyek ꯬, the Extensions) cannot be read.
Licences and data
Network: MIT, like the code. The language model is built from Meitei Mayek Wikipedia (CC BY-SA 4.0) and FineWeb-2 (ODC-By 1.0) and is released under CC BY-SA 4.0, with attribution to those sources. Characters: D. Hijam and S. Saharia, "On developing complete character set Meitei Mayek handwritten character database", The Visual Computer 38, 525–539 (2022).
Demo: huggingface.co/spaces/Chingkheinganba/handwritten-meitei-mayek-word-recognition. Code and every experiment: https://github.com/chingkheinganba231005/meitei-mayek-word-recognition.