Ama Dablam 322M DPO
DPO-tuned Ama Dablam, starting from the instruction-tuned checkpoint: preference-trained to prefer well-formed, on-script, appropriately concise responses over degenerate ones (run-on translations, script drift, near-duplicate low-signal pairs). Trained in two stages — a short DPO-format SFT warmup, then DPO proper (beta=0.1) against a frozen reference — on 14,152 preference pairs across Nepali, Maithili, and Bhojpuri, rendered 85/5/10 across Devanagari/IAST/phonetic script.
| metric | value |
|---|---|
| preference accuracy (val, n=744) | 91.4% (mean margin +4.85) |
| — by language | ne 94.9% · mai 91.0% · bho 87.3% |
| — by script | deva 92.8% · phon 86.3% · iast 77.1% |
| script fidelity (replies in the prompt's script) | 95% (19/20 sampled) |
| response length | mean 32.3 tokens, 95% properly terminated |
Forgetting guardrail (drift vs. the pre-DPO checkpoint on 6 pinned pretrain val shards + phonetic-ne) passed for every metric, but drift was uniformly positive (a small, one-directional erosion, not noise): worst cases IAST-Maithili +0.035 bpb, phonetic-Nepali +0.027 bpb; Devanagari ne/mai/bho drift stayed under +0.01 bpb.
Usage (chat template built in — identical to the instruct model)
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("spandyie/amadablam-322m-dpo")
model = AutoModelForCausalLM.from_pretrained("spandyie/amadablam-322m-dpo",
trust_remote_code=True)
msgs = [{"role": "user", "content": "स्वस्थ रहन के गर्नुपर्छ?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=200)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
Limitations (read before use)
- License: CC-BY-NC-4.0 (non-commercial) — inherited from the instruct checkpoint's Alpaca/Dolly-family lineage, and independently required by the Bhojpuri leg of this model's own DPO data (Bhojpuri routes through NLLB for translation, which is CC-BY-NC-4.0 and propagates to anything built with it).
- 322M params: same factual-reliability caveats as the base/instruct models — this stage tunes response quality/preference, not knowledge.
- Bhojpuri and IAST are the weakest legs (87.3% / 77.1% preference accuracy vs. 91-95% elsewhere) — this tracks thinner/lower-quality source data for those combinations earlier in the pipeline, not a DPO-specific defect.
- Forgetting drift, while within the pass tolerance, moved consistently in one direction across all pinned shards — treat this checkpoint as a first DPO iteration, not a final answer, if you plan to iterate further.
- This model is cacheless by design.
generation_config.jsonsetsuse_cache: false. Overriding touse_cache=Truesilently produces incoherent, context-free output (only the last token is fed back each step) rather than raising an error. - Right-pad with BOS id 16368. No system role. Multi-turn lightly trained.