🇮🇳 Bharat-Tiny-LLM
✨ Why Bharat-Tiny-LLM?
Most Indian-language models do one script. Bharat-Tiny-LLM does both — and does it on hardware that costs less than a pair of shoes.
- 🌐 Truly bilingual — Hinglish and Devanagari Hindi from a single 1.5B model.
- 📱 Edge-native — an 880 MB 4-bit MLX build runs offline on ₹8,000 Android phones & Apple Silicon. No API, no cloud, no internet.
- 🪶 Small but real — 1.5B params, ~57 tok/s on a Mac Mini M4.
- 💸 $0 training cost — fine-tuned on a Mac Mini M4, zero cloud compute.
- 🆓 Open weights — Apache-2.0, self-hostable, no vendor lock-in.
Built by eulogik — an India-first AI lab shipping practical, edge-deployable models.
📊 Model details
| Property | Value |
|---|---|
| Base model | Qwen2.5-1.5B (multilingual, 29 languages) |
| Method | LoRA — 16 layers, rank 16, alpha 32, scale 2.0, dropout 0.05 |
| Trainable params | 10.55M (0.68% of 1.5B) |
| Training data | 436K cleaned Hinglish/Devanagari conversations (gold + NebulaByte + smangrul) |
| Training iters | 110,000 (Mac Mini M4, batch 4, seq 768) |
| Best val loss | 0.937 |
| Quantization | MLX 4-bit affine, group size 64 (~880 MB) |
| Inference speed | ~57 tok/s on Mac Mini M4 |
| License | Apache-2.0 |
🚀 Quick start
Python (recommended)
pip install bharat-tiny-llm[mlx] # Apple Silicon
pip install bharat-tiny-llm[torch] # CPU / CUDA
from bharat_tiny_llm import chat
print(chat([{"role": "user", "content": "Chai peete hain?"}]))
The package ships with a canonical generation config so output is clean out of the box:
temperature=0.3, top_p=0.85, repetition_penalty=1.25, no_repeat_ngram_size=3
MLX CLI
pip install mlx-lm
mlx_lm chat --model eulogik/Bharat-Tiny-LLM
Transformers (fp16 fused variant)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use
temperature ≈ 0.3+repetition_penalty ≥ 1.25
no_repeat_ngram_size = 3. Thebharat-tiny-llmpackage and the demo space do this for you.
llama.cpp (GGUF — Android / Pi / CPU)
# download Q4_K_M GGUF
pip install llama-cpp-python
from llama_cpp import Llama
llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024)
print(llm.create_chat_completion(
messages=[{"role": "user", "content": "Chai peete hain?"}],
temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25,
)["choices"][0]["message"]["content"])
Get the GGUF from eulogik/Bharat-Tiny-LLM-GGUF.
💬 Example outputs
Generated with the 4-bit MLX build at the canonical settings above.
Q: Chai peete hain? A: Haan bilkul! Chai toh humari life hai. Subah uthte hi pehli cheez jo dimag mein aati hai woh chai hi hoti hai...
Q: Kal interview hai, nervous ho raha hoon A: Okay chill karo yaar! Interview mein sabse pehle confidence dikhana hota hai. Deep breath lo, apna intro clear rakho...
Q: Biryani kaise banate hain? A: Biryani ek dum simple process hai, jismein kuchh ingredients hote hain aur cooking time bhi kam hota hai. Sabse important cheez hai ingredients...
Q: Ghar baithe paise kaise kamayein? A: Haan, bilkul sahi sawaal hai! Ghar baithe paise kaise kamayein? Yeh toh bahut common sawaal hai aaj kal. Tension nahi lene ka, main tumhe seedha-seedha points mein batata hoon...
Q: भारत के कुछ प्रसिद्ध त्योहार बताओ। A: भारत के प्रसिद्ध त्योहार हैं: दीवाली, होली, रक्षा बंधन, ईद, नवरात्रि और बहुत से और। हर त्योहार का अपना अलग महत्व है...
🗂️ Model variants
| Repo | Format | Size | Use |
|---|---|---|---|
eulogik/Bharat-Tiny-LLM | MLX 4-bit | ~880 MB | Edge / Apple Silicon (this repo) |
eulogik/Bharat-Tiny-LLM-GGUF | GGUF Q4_K_M | ~1.06 GB | Cross-platform (llama.cpp, Android, Pi, CPU) |
eulogik/Bharat-Tiny-LLM-fused | PyTorch fp16 | ~3.3 GB | Server / fine-tuning base |
🧩 Limitations
- 1.5B params: great for short conversational replies, weaker on long factual essays or complex reasoning.
- Trained primarily on Hinglish + Hindi; other Indian languages are inherited from the Qwen2.5 base, not fine-tuned.
- Quantization (4-bit) trades a little fluency for a 4× size reduction.
- May occasionally repeat or drift on very long generations — keep
max_new_tokensmodest and use the n-gram guard.
🛣️ Roadmap
- More Indic languages (Tamil, Telugu, Bengali, Marathi)
- DPO / preference tuning for a quality leap
-
BharatTiny-Bench— an Indic edge-model evaluation suite - GGUF build for broader Android / Raspberry Pi deployment
🔗 Links
- 🤗 Model card: https://huggingface.co/eulogik/Bharat-Tiny-LLM
- 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
- 🤗 fp16 fused: https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused
- 🚀 Demo space: https://huggingface.co/spaces/eulogik/bharat-tiny-llm-v3-demo
- 💻 Source code: https://github.com/eulogik/Bharat-Tiny-LLM
- 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
- 🏢 Built by eulogik
📜 License
Apache-2.0. Base weights Qwen2.5-1.5B are Apache-2.0; the LoRA adapter is Apache-2.0.