Bharat-Tiny-LLM (fused · fp16)
This is the full-precision fused model for Bharat-Tiny-LLM —
the LoRA adapter merged into the base Qwen2.5-1.5B weights,
in PyTorch float16.
Use this repo when you want to:
- run inference on CPU / CUDA with
transformers, - fine-tune further, or
- produce your own quantized builds (GGUF, MLX, etc.).
Built by eulogik
For most users
You probably want a smaller, ready-to-run build instead:
| Build | Repo | Size | Use |
|---|---|---|---|
| MLX 4-bit (edge / Apple Silicon) | eulogik/Bharat-Tiny-LLM | ~880 MB | Recommended for Mac / on-device |
| GGUF Q4_K_M (llama.cpp, Android / Pi / CPU) | eulogik/Bharat-Tiny-LLM-GGUF | ~1.06 GB | Cross-platform, llama.cpp |
| PyTorch fp16 (this repo) | eulogik/Bharat-Tiny-LLM-fused | ~3.3 GB | Server / fine-tuning base |
Quick start (transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
messages = [{"role": "user", "content": "Chai peete hain?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.3,
top_p=0.85,
repetition_penalty=1.25,
no_repeat_ngram_size=3,
do_sample=True,
)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use
temperature ≈ 0.3+repetition_penalty ≥ 1.25+no_repeat_ngram_size = 3. Thebharat-tiny-llmPyPI package applies these for you.
Links
- 🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
- 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
- 🚀 Demo: https://huggingface.co/spaces/eulogik/bharat-tiny-llm-v3-demo
- 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
- 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
- 🏢 Built by eulogik
License
Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).