Mtrini-27B-Tellus-Merged
This repository contains the merged Transformers model for Mtrini-27B-Tellus.
The CompiwerAI LoRA adapter has been merged into the compatible base model, producing a standalone model for Transformers-based inference.
CompiwerAI — Building AI For Everyone.
Model Information
| Property | Value |
|---|---|
| Model | Mtrini-27B-Tellus |
| Type | Merged Transformers model |
| Base | Qwen3.8-27B |
| Parameters | ~27B |
| Context | 4096 tokens |
| Training steps | 700 |
| Training method | QLoRA / LoRA |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| Learning rate | 0.00015 |
| Compute dtype | BF16 |
Training Mix
- OpenCodeInstruct — 35%
- Magicoder — 10%
- OpenR1-Math — 25%
- Moroccan Darija — 20%
- Aya Arabic / Moroccan Arabic — 10%
Total: 11,200 examples
Training Result
- Final loss: 0.4302458722250802
- Runtime: approximately 3.15 hours
- Epoch: 1
- Hardware: NVIDIA RTX PRO 6000 Blackwell Server Edition
Transformers Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "CompiwerAI/Mtrini-27B-Tellus-Merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
For large-model inference, substantial GPU or CPU memory may be required.