Claudius 400M Chat: a Latin conversation tutor
Claudius is a Latin teacher you can chat with, in Latin only. It answers briefly, keeps the conversation going with a question, adapts to the learner's level, copes with a beginner's broken Latin, and can correct a learner's sentence. Claudius has a fixed persona: he lives in Rome near the Forum, has a sister, Claudia, and a dog, Argus, and loves Vergil.
DISCIPULUS: Salvē, magister! Quis es?
MAGISTER: SUM CLAUDIUS, MAGISTER TUUS. QUID HODIE DISCERE UIS?
DISCIPULUS: Hodie pluit.
MAGISTER: ITA, MULTUM PLUIT. HABESNE UMBELLAM?
DISCIPULUS: Tu nomus?
MAGISTER: NOMEN MEUM EST CLAUDIUS. QUOD EST NOMEN TUUM?
DISCIPULUS: Duos fratres habeo, Stipe et Dario. Et tu?
MAGISTER: EGO FRATRES NON HABEO, SED SOROREM CLAUDIAM. SUNTNE FRATRES TUI MAIORES AN MINORES?
This is a fine-tune of fbandov/claudius-latin-400m, a 342M-parameter model pretrained from scratch on Latin text only. It has never seen English.
Chat
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("fbandov/claudius-latin-400m-chat")
model = AutoModelForCausalLM.from_pretrained("fbandov/claudius-latin-400m-chat")
messages = [
{"role": "system", "content": "tiro"}, # level: tiro | medius | peritus
{"role": "user", "content": "Salvē, magister! Quis es?"},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt", return_dict=True)["input_ids"]
out = model.generate(ids)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True).strip())
- Levels. The system message picks the learner's level:
tiro(beginner),medius(intermediate) orperitus(advanced). The chat template turns it into the fixed Latin instruction the model was trained on. Any other system text is ignored. - Any Latin input. The tokenizer normalises by itself: accents and macrons are dropped, letters uppercased, V becomes U and J becomes I.
- Output is in that orthography. For display, write U as V (SALVE, VERBVM).
- Defaults are set for chat. Temperature 0.5, top-p 0.9, repetition penalty 1.15, no repeated 4-grams, 60 new tokens. Generation stops at the end of the teacher's turn.
Correcting a sentence
The chat does not correct errors on its own; that is by design. A 400M-parameter model that corrects in the middle of conversation sometimes "corrects" Latin that was right, and a tutor that mis-corrects is worse than none. Corrections are a separate, checkable step:
def correct(sentence: str) -> str:
ids = tok(f"CORRIGE: {sentence}\nRECTE:", return_tensors="pt").input_ids
# Copying the sentence is most of the task, so switch the chat
# defaults' repetition penalties OFF here.
out = model.generate(ids, do_sample=False, repetition_penalty=1.0, no_repeat_ngram_size=0)
return tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True).strip()
correct("Puella rosam amant.") # PUELLA ROSAM AMAT.
correct("Milites urbem defendit.") # MILITES URBEM DEFENDUNT.
correct("Ego librum legit.") # EGO LIBRUM LEGO.
correct("Puella rosam amat.") # PUELLA ROSAM AMAT. (correct: unchanged)
If the output differs from the input, start the teacher's reply with the fix and let the model carry on. It was trained on replies that open this way:
messages = [{"role": "system", "content": "tiro"},
{"role": "user", "content": "Puella rosam amant."}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
prompt += ' SED DICENDUM EST "AMAT", NON "AMANT".'
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True).strip())
# e.g. ITA. UBI ROSAS AMAS?
Correction quality is in the evaluation below. For firm corrections, the Claudius app also requires agreement from a second detector and a morphological parser (LatinCy); flag model-only fixes as a question ("NUM X DICERE UOLUISTI?") rather than a verdict.
Training
- Base: claudius-latin-400m (Llama architecture, 24 layers, hidden 1024, 342M parameters; pretrained on 6.84B tokens of Latin, then annealed on classical Latin and Vicipaedia).
- Conversations: 16,491 synthetic teacher-student dialogues in Latin across three levels. The teacher has a persona (Claudius); 61% of dialogues open with small talk; every teacher turn is one or two sentences ending in one question. The loss is on the teacher's words only. Half the dialogues contain a deliberately corrupted student turn followed by the teacher's correction; the correction text is context (not a training target), so the model learns to continue after a fix but never to invent one.
- Persona: 4,732 dialogues in which the student asks about Claudius's own life ("ET TU?", "UBI HABITAS?") and names several people, answered from one fixed persona. They train I/you and mine/yours, and plural agreement.
- Broken input: 2,833 dialogues in which a beginner writes broken Latin (misspellings, missing verbs, word salad, "TU NOMUS?"), and Claudius works out what was meant, answers it, and recasts it correctly.
- Corrections: 37,770 learner errors of 8 kinds (subject-verb agreement, person, tense, irregular forms, case after prepositions, object case, adjective agreement, noun endings), made from correct sentences in the Vulgate, classical Latin and the dialogues. The fix is always the original, correct sentence. Correct sentences are copied unchanged, so "no change" is the usual answer.
- 443 steps, batch 24 x 1,024 tokens, learning rate 5e-5.
Evaluation
Conversation. 51 fixed learner messages (greetings, identity questions, small talk, and 21 messages with errors), temperature 0.5, each reply graded for whether it is on topic:
| identity questions answered as Claudius | 7 / 7 |
| off-topic replies | 8% |
| repetition loops | 0% |
Persona and perspective. 23 multi-turn cases (personal questions, "ET TU?", several named people, and real beginners' broken messages), 3 samples each, graded for I/you perspective, consistency with the persona, the student's names, and whether the reply answers what a broken message meant:
| perspective | 93% |
| persona consistent | 99% |
| names right | 96% |
| sensible reply to the message | 71% |
| all four at once | 71% |
Corrections, with the correct() function above (any change counts as a flag):
| test set | correct sentences wrongly changed | errors flagged | exact fix |
|---|---|---|---|
| 51 learner messages (29 correct, 21 with an error) | 1 / 29 | 19 / 21 | 17 / 21 |
| learner errors in literary sentences, never trained on (1,000 + 1,000) | 9.3% | 62% | 48% |
| held-out Vulgate verses, two sets (300 + 200 each) | 8.7% / 7.0% | 62% / 54% | 48% / 36% |
On the short, simple sentences a learner writes it is accurate. On long literary sentences it changes about 1 correct sentence in 11, so show a model-only change as a question, not a verdict. The Claudius app's stricter pipeline (a firm correction only when a second detector and the LatinCy parser agree) is wrong on 0 of 1,000 and 1 of 300 correct sentences, and still catches 57-59% of the Vulgate errors.
Limitations
- Occasional echo. On short statements it sometimes asks back what the student just said ("EGO CANEM HABEO." -> "TU CANEM HABES?"), about 6% of replies. The Claudius app resamples those; a simple check is to compare the reply's closing question with the student's statement.
- Short conversations about everyday things. It was trained on dialogues about daily life, school, family, food, the weather and Roman topics. Go far beyond that and it drifts.
- Not a source of facts. At this size it states things that are wrong (it once put 50,000 soldiers in a legion). Treat its facts as conversation, not information.
- Corrections are a helper, not an authority. On simple learner Latin they are accurate. On long literary sentences they miss about half of the errors and change about 8% of correct sentences.
- Uppercase classical orthography only, and no Greek.
License
CC BY-SA 4.0 (the base model's training text includes CC BY-SA sources). Credit "Claudius (fbandov)".