Technical Report: Ntahokaja 0.31 (Base)
Ntahokaja 0.31 is the first sovereign Small Language Model (SLM) trained from scratch specifically for the Kirundi language. Developed by Nyehashi AI, it features a custom morphological tokenizer designed to respect Bantu grammar and agglutinative logic.
Quick Start (Usage)
This model requires custom tokenization logic. You must use trust_remote_code=True to load it correctly.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Nyehashai/Ntahokaja-0.31-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id)
prompt = "Amategeko yu'Burundi avuga ko "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0]))
Section 1 — Executive Summary & Architecture
Ntahokaja 0.31 is a 91,209,216-parameter Transformer-based language model. Unlike general models, it uses a Hybrid Morphological-BPE strategy. This version resolves the "word-boundary collapse" by using an explicit @@ sub-word joiner and ByteLevel space protection. Layers: 12 Attention Heads: 12 Context Window: 512 tokens Size: 347.9 MB (Safetensors)
Section 2 — Performance
Metric Value Validation Loss 3.24 (Optimal) Training Loss 1.26 Vocabulary Size 7,500 tokens Data Volume 18.1M tokens
Section 3 — Behavioral Observations
The model demonstrates high-fidelity knowledge of Burundian geography (identifying provinces like Makamba, Kirundo, etc.) and political history. Notably, the model exhibits Implicit Moral Alignment, pivoting away from harmful or violent prompts toward themes of social respect and constitutional law.
Section 4 — Intended Use & Limitations
This is a Foundational Base Model. It is designed for text continuation, linguistic research, and as a base for instruction fine-tuning. It is NOT a conversational assistant and has not been fine-tuned for QA yet.
Citation and Attribution
If you use this model, please attribute the work to Nyehashi AI:
Lead Engineer: Ishimwe Lilas
Additive Current Data : Iradukunda Cherina, Itegineza Alvin
Economic Advisory: Igiraneza Charine
Organization: Nyehashi AI (Burundi)