🌱 Mavuno-195M-Instruct

Mavuno-195M is a domain-specialized Small Language Model (SLM) trained from scratch on STEM and East African agronomy literature.

Model Summary

• Parameters: 195,008,640 (~195M)
• Architecture: LLaMA-based (RMSNorm, RoPE, SwiGLU, Weight Tying)
• Context Length: 2,048 tokens
• Pretraining Tokens: 5.25 Billion tokens (STEM & Agriculture)
• Fine-Tuning: 50,711 curated SFT pairs (MetaMathQA, SciQ/ARC/OpenBookQA, CodeAlpaca, Agriculture-QA)

Benchmark Results

Evaluated on full standard benchmark test sets:

• SciQ (Test Split): 62.10%
• ARC-Easy (Test Split): 47.98%
• OpenBookQA (Test Split): 30.40%
• MMLU Elementary Math: 32.28%
• MMLU HS Biology: 30.00%

Quick Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Dynamoabey/Mavuno-195M-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

prompt = "System: You are Mavuno, an expert STEM and agricultural AI assistant.\n\nUser: How do farmers enrich nitrogen-deficient soil using legume intercropping?\n\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use & Safety Notes

Mavuno-195M is an edge-optimized scientific demonstrator. Like all sub-1B parameter models, multi-step symbolic calculations and veterinary diagnoses should be verified with external tools or retrieval-augmented generation (RAG).