Model card: AnuLM-Hindi-QA-400M (ckpt_multi_qa.pt)
A Hindi / English / Python question answerer in the encyclopaedia register:
the three-language AnuLM base (AnuLM-Base-400M, step 36,000) fine-tuned
on question → lead-paragraph pairs built from Wikipedia and from Python
docstrings. Full log and decoded samples: docs/RESULTS.md §23.
Licence: CC BY-SA 4.0. The pairs derive from Hindi and English Wikipedia (CC BY-SA); the base model also saw C4 (ODC-BY) and GitHub Python via codeparrot-clean (mixed licences). Not affiliated with Sarvam AI, AI4Bharat, BharatGen or the Government of India.
What it does, and does not
Ask X क्या है?, X के बारे में बताइए।, What is X? or What does this function do? and it answers with one or two definition-style sentences.
It is not a chatbot: it has never seen a conversation, has no
identity, and treats "What is your name?" as a topic to define. It invents
details freely; check anything that matters.
Held-out answer loss 3.3150 at the selected step. Manual scoring in
docs/RESULTS.md §23: sensible, on-topic answers for well-known subjects
in all three languages, degrading quickly on rare ones.
Data
| pairs | built by | from |
|---|---|---|
| Hindi questions | make_qa.py | Hindi Wikipedia lead paragraphs (CC BY-SA) |
| English questions | make_qa_en.py | English Wikipedia lead paragraphs (CC BY-SA) |
| Python questions | make_qa_py.py | docstrings of functions in codeparrot-clean |
Prompt formats are stored in the checkpoint as qa_templates:
प्रश्न: {q} newline उत्तर: for Hindi and Question: {q} newline
Answer: for English and Python; loss on the answer tokens only.
Architecture and tokenizer
Same network as every 400M AnuLM checkpoint: 20 layers, hidden 1,024,
GQA 16 query / 4 key-value heads, 24 routed experts of 192 with top-4
routing and aux-loss-free balancing, sliding window 256 on layers 0–9,
context 512, 32k vocabulary. 397.7M parameters, 173.5M active per token.
Tokenizer multi32k (docs/RESULTS.md §22).
Use
python serve.py --ckpt <this folder> then open http://127.0.0.1:8000
and pick answer a question, or python ask.py --ckpt <this folder> on
the command line. Temperature 0.3 and repetition penalty 1.3 are the
settings the samples were produced with.
How to load
With transformers
The modelling code travels with the weights, so trust_remote_code=True is
required and there is nothing to clone:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("toonist/AnuLM-Hindi-QA-400M", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("toonist/AnuLM-Hindi-QA-400M")
ids = tok("\u092a\u094d\u0930\u0936\u094d\u0928: \u0915\u093e\u0936\u0940 \u0915\u094d\u092f\u093e \u0939\u0948?\n\u0909\u0924\u094d\u0924\u0930:", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=False)[0]))
Greedy output is identical to the AnuLM repository's own generate, cached or
not, and the tokenizer here agrees with bpe.py token for token; both are
pinned by tests in that repository.
Load it in float32 — which the config now asks for, so the line above is
enough. Do not force dtype=torch.bfloat16: this is a mixture-of-experts model
whose router keeps a per-expert bias, and rounding that bias to 16 bits changes
which experts fire. The output does not get slightly worse, it collapses into
repeated tokens. The big tensors are stored in bfloat16 and upcast on load; the
router bias and the norms are stored in float32 for this reason. For speed, use
torch.autocast over float32 weights, which is how every number in this card
was measured. The model loads the code from this repo at
trust_remote_code=True; pin a revision if you want that fixed.
Batches must be unpadded (one sequence at a time), and beam search is not supported.
With the AnuLM repository
Every script there takes this folder wherever it takes a .pt:
hf download toonist/AnuLM-Hindi-QA-400M --local-dir AnuLM-Hindi-QA-400M
python serve.py --ckpt AnuLM-Hindi-QA-400M # web page at http://127.0.0.1:8000
python sample.py --ckpt AnuLM-Hindi-QA-400M --prompt "\u092a\u094d\u0930\u0936\u094d\u0928: \u0915\u093e\u0936\u0940 \u0915\u094d\u092f\u093e \u0939\u0948?\n\u0909\u0924\u094d\u0924\u0930:"
from model import load_checkpoint, AnuLM
ck = load_checkpoint("AnuLM-Hindi-QA-400M")
m = AnuLM(ck["cfg"]).eval(); m.load_state_dict(ck["model"])
Files: model.safetensors (397.7M parameters), tokenizer.multi32k.json for bpe.BPE.load,
and tokenizer.json in the tokenizers format for AutoTokenizer.