IrisSpeak-135M
IrisSpeak-135M is a context-aware next-card model for augmentative and alternative communication (AAC). Given the setting, the recent conversation, the communication partner's last utterance and the cards a user has already tapped, it scores every card in a closed AAC vocabulary and predicts which card comes next. It runs entirely on the device: in the browser through onnxruntime-web, or in PyTorch.
Live app: https://irisspeak.org/ (no account needed).
Architecture
- Base: SmolLM2-135M-Instruct (30 layers, hidden 576, Apache 2.0).
- Card layer: a trainable matrix with one row per vocabulary card plus a name slot, a start marker and an end-of-message marker (3,097 rows). The rows are used both as input embeddings for the tapped cards and as the output head that turns the last hidden state into one logit per card.
- Each card row was initialised from the mean SmolLM2 embedding of the card's spoken form, then the whole model was fine-tuned on next-card states derived from AAC conversation data.
Files
| File | What it is |
|---|---|
model.safetensors, config.json | LlamaForCausalLM with the card rows folded into the vocabulary: card i is token id 49152 + i; the start marker is token 52248 (see extended_vocab.json). Base vocabulary 49,152, extended 52,249. |
tokenizer.json and friends | the unchanged SmolLM2 tokenizer for the text part of the prompt |
extended_vocab.json | ordered card ids (card_0000 …, <name>, <aac_end>) mapped to the extra token ids |
onnx/card_model_fp16.onnx | the same graph exported with Optimum, fp16 weights (274 MB); inputs input_ids, attention_mask, position_ids; top-100 overlap with PyTorch 0.998 |
card_model.pt | the original training checkpoint (state dict, fp32) |
The card vocabulary itself (the spoken label of each card id) is not part of this repository.
Prompt format
Setting: home.
Earlier: <partner> | <answer> | ...
Partner: What do you want for lunch?
Reply cards:
followed by the start token and the token ids of the cards tapped so far. Take the logits of the last position restricted to the card token range and softmax them to get a distribution over the next card.
Training data
Next-card states were derived from public AAC conversation corpora (AAC Conversations, AACText imagine) mapped onto the card vocabulary: 69,676 training states, weighted towards question-answer exchanges, with a six-card cap per message and the end-of-message marker down-weighted.
Results (held-out states)
| metric | value |
|---|---|
| Recall@1 | 0.29 |
| Recall@16 | 0.55 (0.60 on question-answer states) |
| Recall@100 | 0.78 |
| Recall@16 with the trained reranker | 0.65 general / 0.70 question-answer |
Intended use
Research prototype for context-aware AAC card suggestion. Not a medical device.