Sahaj-86M (SEHAJ v8)
An 86M-parameter Punjabi language model — hybrid Selective-SSM + causal- attention architecture, trained from scratch on a curated Punjabi corpus, paired with a retrieval-augmented (RAG) factual-QA layer and a trained confidence/abstention probe.
Successor to Nam-toon-studio/sahaj-30m — kept as a separate, larger sibling model, not a replacement. The two have different, real trade-offs (see below); pick based on your constraint (smallest footprint vs. best factual accuracy).
Code to run it: https://github.com/gurpreetsingh5523-source/sahaj-86m
What this is
- Text-only. No audio, no vision, no code generation.
- Punjabi-first, not an English or multilingual conversational model.
- Meant for local/on-device/low-resource deployment.
Honest numbers
| Sahaj-86M (this) | Sahaj-30M (previous) | |
|---|---|---|
| params | 86,036,224 | 30,417,152 |
| pretrain val_loss | 2.86 | 3.10 |
| raw factual QA (no RAG), early 25-question set | 12% | 4% |
| raw factual QA, 143-question decontaminated set (2026-09-26) | 25.9% | not measured |
| factual QA (with RAG) | ~40% (retrieval-only, same for both) | ~40% |
| EOS stop rate | 50% | 60% |
| confidence-head AUROC | 0.70 | — |
The 86M model isn't a strict upgrade: it recalls more facts unaided (12% vs 4% on the early set) but has a slightly worse EOS-stop rate and general-corpus perplexity after fine-tuning — it specializes harder on the instruction-following format it was trained on. Pick 30M for the smallest footprint, 86M for better raw factual recall.
Prompt format matters. Trained on the literal template
"ਸਵਾਲ: {prompt}\nਜਵਾਬ: " — send prompts through that template (the reference
server code does this automatically) for meaningfully more coherent output
than bare/informal prompts.
Capacity note: at this scale, factual world knowledge is carried by the RAG layer (retrieval over a local Wikipedia-pa index), not memorized in the weights. Use this as a fast, private conversational layer with a grounded lookup tool behind it.
Files
sft_checkpoint_best.pt— full checkpoint (model + optimizer state), what the referenceinference.py/server.pyload out of the box.model_fp16.safetensors+config.json— smaller weights-only export (~181MB) for anyone building their own loader.sevak_bpe_6k.model— the SentencePiece BPE tokenizer (vocab 6000).
Run it
git clone https://github.com/gurpreetsingh5523-source/sahaj-86m
cd sahaj-86m
pip install -r requirements.txt
# download sft_checkpoint_best.pt and sevak_bpe_6k.model from this repo into the same folder
python3 inference.py --checkpoint sft_checkpoint_best.pt --prompt "ਸਵਾਲ: ਸਤਿ ਸ੍ਰੀ ਅਕਾਲ\nਜਵਾਬ:"