Halfword 100M
A 0.1 B-parameter Llama-architecture language model for AAC word prediction, trained to run on the device and paired with a decoder that optimises seconds to utterance rather than keystrokes saved.
The model on its own is unremarkable — a small LM trained on public conversation. What it is for is the interesting part: it feeds a decoder that knows what a candidate costs to reach on the person's actual access method, and chooses the candidate list accordingly.
- Code, benchmark and figures: https://github.com/NagaYu/halfword
- Interactive demo: https://huggingface.co/spaces/NagaYu/halfword
- Benchmark dataset: https://huggingface.co/datasets/NagaYu/halfword-bench
What ships here
| file | what it is |
|---|---|
model.safetensors, config.json | the 101 M-parameter Llama-architecture model |
tokenizer.json | 16 k byte-level BPE, trained on the same corpus |
halfword.onnx | ONNX export, dynamic batch and sequence, verified in onnxruntime |
halfword.mlpackage | Core ML export, float16 |
halfword.gguf | GGUF under the stock llama architecture, for llama.cpp |
ngram.json | 4-gram fallback predictor — runs with no accelerator and no torch |
calibrator.json | acceptance calibrator (see below; the decoder does not work without it) |
Architecture
Llama-architecture (RMSNorm, RoPE, SwiGLU, tied embeddings). That is an export decision, not a preference: it is what Core ML, ONNX and llama.cpp's GGUF path all already understand, so one definition leaves in three formats without a bespoke converter for each.
| parameters | 100.7 M |
| layers / hidden / heads / KV heads | 14 / 768 / 12 / 4 |
| intermediate | 2048 |
| vocabulary | 16 384 |
| context | 512 |
Usage
from halfword.predictor import NeuralPredictor, Context
pred = NeuralPredictor.load("NaGaYu/halfword-100m") # local dir or snapshot
pred.warmup()
for c in pred.predict(Context(history=("how are you today",), committed="i would like some wa")):
print(c.insert, round(c.probability, 4), c.kind)
Predictions are prefix-constrained: having typed wa, every candidate extends wa. Tokens that cannot are masked before the softmax is inspected, which both improves the candidates and bounds the work — a word finishes in one or two tokens far more often than a free continuation does.
To get the point of the project, use it with the decoder:
from halfword.costs import CostModel, Surface, load_keyboards
from halfword.decoder import CostAwareDecoder
from halfword.calibrate import AcceptanceCalibrator
cost = CostModel.from_name("scanning", scan_interval_s=1.2)
surface = Surface(load_keyboards()["abc_6x6"])
decoder = CostAwareDecoder(max_candidates=8,
calibrator=AcceptanceCalibrator.load("calibrator.json"))
shown = decoder.present(pred.predict(ctx), surface, cost) # may legitimately be 0 candidates
How good is it? Worse than the n-gram beside it.
Measured on 8 000 utterances the model never saw (rows beyond the training cut of each source), top-5, 120 word boundaries:
| next word, nothing typed | word completion, 2 letters typed | latency | |
|---|---|---|---|
ngram.json (4-gram, 15 MB) | 59.2% | 90.2% | 0.7 ms |
| this model (101 M) | 9.2% | 63.9% | 197 ms |
That is not a typo and it is not a bug hunt left unfinished. The trivial baseline shipped alongside this model beats it comfortably, and you should use ngram.json unless you have a reason not to.
The reason is ordinary: the model saw roughly 9 M tokens of training — about three passes over a 3 M-token corpus, in 100 minutes on a laptop GPU. Small language models that are actually good see three to four orders of magnitude more. This one has learned English morphology (pl → please, wi → with, m → much) but not much about what people say next, which is exactly the shape you would predict from the token budget.
What it is good for. It is a working reference for the parts that are hard to get right and easy to get silently wrong: the Llama-architecture sizing, the three exports with their gotchas, the prefix-constrained decoding with token healing, and the KV-cache reuse that keeps latency inside a scan step. All of that is correct and tested. Swap in better weights and the rest of the stack does not change.
What would fix it. More tokens, in this order: a larger and more conversational corpus (the current mix is 40% Tatoeba single sentences, which teach grammar but not dialogue), 10–50× the training steps, and a held-out early-stopping signal instead of a fixed step count. None of that is research; it is compute.
Latency
The calibrator is not optional
The decoder converts probabilities into seconds. A raw model score is not "the probability this candidate is what the person means", and fed raw scores the decoder concludes prediction is always right — which makes long candidate lists free and collapses the entire trade-off the project exists to model. calibrator.json fits "is this the first correct candidate in the pool", so the residual mass is a genuine "you will have to spell another character". Held-out expected calibration error is reported in provenance.json.
Training data
Public, openly-licensed conversational corpora only. Licences were checked individually and are recorded per source with a verification date.
| source | licence |
|---|---|
allenai/soda | CC BY 4.0 |
google/Synthetic-Persona-Chat | CC BY 4.0 |
pfb30/multi_woz_v22 | Apache-2.0 |
Helsinki-NLP/tatoeba | CC BY 2.0 FR |
No AAC user data. There is, as far as we know, no public openly-licensed corpus of real AAC communication, and this project did not create one. The register of open-domain chat is not the register of AAC use, and that gap is a real limitation of everything here.
Non-commercial corpora (e.g. DailyDialog, CC BY-NC-SA) are excluded by default. Including them would make this model non-commercial too.
Limitations
- It loses to a 4-gram. See the table above. This is a reference implementation with real weights, not a good predictor, and the card says so where you will read it rather than in a footnote.
- Small model, narrow corpus. 0.1 B parameters, 3 M training tokens, 100 minutes. It will not know your family's names, your medications, or your jokes — and on this evidence it does not know much about ordinary conversation either. On-device personalisation addresses the first of those, and never sends anything anywhere.
- The seconds are simulated. Nobody has been timed using this. The cost models price mechanical scan, dwell and pointing time under stated assumptions.
- Visual search cost is not modelled. The AAC literature argues the cost of reading a prediction list is a large part of why prediction can slow people down. Ignoring it makes long lists look cheaper than they are, so the reported gaps are a lower bound.
- Every duration is a property of an interface, never of a person. A long scan interval means this interface costs more per selection and nothing about who is using it.
If you use AAC, or support someone who does
This was built without AAC users in the room, which is a real limitation and not a small one. Blasko et al. (2025) put the problem in a title: "Nothing about AAC users without AAC users". If the defaults look wrong to you — the scan intervals, the dwell times, the assumption that seconds are the right thing to minimise at all — please open a discussion here or an issue on GitHub. We would rather be corrected than cited.
Citation
The finding that keystroke savings and communication rate come apart is not ours. See Koester & Levine (1994, 1996), Higginbotham (1992), Trnka et al. (2007–2009), Quinn & Zhai (2016), and especially Cai et al. (2024, Nature Communications). What is new here is narrower: identifying positional cost rather than slowness as the mechanism, and making candidate-list length a computed decision variable in a time objective. Full positioning in the repository README.