Laabam AI 3B v1
A multilingual AI assistant fine-tuned from Qwen2.5-3B-Instruct using QLoRA.
Training Details
- Base model: Qwen2.5-3B-Instruct (4-bit quantized)
- Method: QLoRA (r=16, alpha=32)
- Training: 4 epochs on ~98K samples (final train loss 0.465)
- Languages: English, Hindi, Telugu, Kannada, Tamil
- Domains: General instruction following, coding, reasoning, safety alignment, Indic languages
Training Epochs
| Epoch | Dataset Size | Learning Rate | Focus |
|---|---|---|---|
| 1 | 36K | 2e-4 | Core instruction following |
| 2 | 36K | 5e-5 | Continued refinement |
| 3 | 98K | 2e-5 | Expanded: safety, Indic languages, clean instructions |
| 4 | 98K | 1e-5 | Careful refinement (low LR, anti-forgetting) |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("laabamone/laabam-ai-3b-v1")
tokenizer = AutoTokenizer.from_pretrained("laabamone/laabam-ai-3b-v1")
messages = [{"role": "user", "content": "Hello, who are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
License
Apache 2.0