ੴ Sehaj 30M

ਸਹਿਜ 30M — a small byte-level Punjabi language model

By Gurpreet Singh Dhillon (Nam-toon Studio).

Sehaj 30M is a small (~29.6M parameter) experimental language model that reads and writes raw UTF-8 bytes (vocab 256, no tokenizer), aimed mainly at Punjabi (Gurmukhi). It is a research model, small enough to run on a laptop CPU.

Card corrected (2026-10-01). An earlier version of this card listed benchmark numbers (healthcare Q&A 84.6%, instruction following 91.4%, grammar 89.2%, phone-chip latency tables, baseline comparisons) that were never actually measured. They have been removed. Treat this as an early experiment, not a tested assistant.

What it is

• Architecture: causal byte-level transformer (SEHAJ v4.1 lineage): 6 layers, d_model 640, 8 heads / 2 KV heads (GQA), SwiGLU, RoPE, a fast-weight attention variant, and iterative "vichar" refinement steps (3 in training, up to 5 at eval). See config.json and modeling_sahaj.py.
• Context: 512 bytes.
• Fine-tuning: question-answer instruction fine-tune (m30_qa_finetuned.pt).

Honest limitations

• Not reliable for facts. At this size, models of this family recall very few facts unaided, so answers are often wrong. Do not use it for medical, legal or other important decisions.
• Repetition. Small models like this often loop or repeat lines; use sampling with a repetition penalty.
• No benchmarks yet. We have not run a proper held-out evaluation of this checkpoint.
• For better factual accuracy, see the larger sibling Nam-toon-studio/sahaj-86m, which has measured numbers and a retrieval (RAG) layer.

Files

• m30_qa_finetuned.pt: fine-tuned PyTorch checkpoint
• modeling_sahaj.py: model definition
• generate.py: generation script
• config.json: hyperparameters
• sehaj_omni_service.py, app.py: experimental local server / demo code
• SahajAI-30M.apk: an early experimental Android app build. It has not been security-reviewed or evaluated, and it is not known to run this model on the phone; treat it as a UI prototype.

Quick run

git clone https://huggingface.co/Nam-toon-studio/sahaj-30m
cd sahaj-30m
python3 generate.py

License

MIT.