bhai-lang-coder-1.5b
A Qwen2.5-Coder-1.5B-Instruct model fine-tuned to write bhai-lang — the toy programming language with Hindi-slang keywords (hi bhai, bol bhai, agar bhai, jab tak bhai, …).
Ask it in English or Hinglish; it replies with a runnable bhai-lang program:
hi bhai
bhai ye hai n = 6;
bhai ye hai result = 1;
bhai ye hai i = 1;
jab tak bhai (i <= n) {
result = result * i;
i = i + 1;
}
bol bhai result;
bye bhai
Results
- 100/100 functional pass rate on a held-out eval: each generated program was executed by a bhai-lang interpreter and its stdout compared against a reference solution. Held-out tasks used unseen numbers, strings, and phrasings across all 27 trained task families.
- Partial generalization to task types absent from training (3/5): it composed known idioms into new programs (sum of odds via step-2 loops, digit counting), and failed in instructive ways (a fencepost bug; hallucinating
nahi to {for a bare else).
Training
- Base:
mlx-community/Qwen2.5-Coder-1.5B-Instruct-4bit - Method: QLoRA (r=8, alpha=16, 16 layers), 3 epochs, lr 1e-5, prompt-masked SFT — trained locally on an Apple M1 Pro (16 GB) in ~40 minutes with Troy
- Data: 1,400 synthetic instruction→program pairs from 27 parameterized task templates (print/arithmetic/conditionals/loops/break-continue/strings/booleans through GCD, primality, Fibonacci, digit manipulation), with English and Hinglish instruction phrasings. Every training program was execution-verified: run through a bhai-lang interpreter and checked against an independently computed expected output before inclusion.
Usage (MLX / Apple Silicon)
from mlx_lm import load, generate
model, tokenizer = load("avirajkhare00/bhai-lang-coder-1.5b")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Write a bhai-lang program that prints the factorial of 6."}],
add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt, max_tokens=300))
Prompting notes:
- Send a bare user turn (no system message) — the model was trained without one, and adding one degrades format adherence.
- Extract the program with the regex
hi bhai[\s\S]*?bye bhai; generation may not stop cleanly at end-of-turn, so trailing tokens afterbye bhaishould be discarded.
Intended use & limitations
A hobby/research artifact demonstrating that a small locally-trained model can learn a niche programming language from synthetic, execution-verified data. It covers bhai-lang's core (variables, printing, if/else-if/else, while, break/continue, arithmetic/comparison/logic, strings, sahi/galat/nalla). It does not know functions or arrays (neither does core bhai-lang), and can slip on algorithm shapes far from its training distribution.