Gonyai-v1: A Poetic Konkani Language Model
Gonyai-v1 is a 160M parameter transformer model specifically designed for Konkani text generation. It features a custom architecture (KonkanGPT) utilizing Rotary Positional Embeddings (RoPE), RMSNorm, and SwiGLU activation functions.
Unlike general-purpose models, Gonyai-v1 is a linguistic specialist focused on the cultural and poetic nuances of the Konkani language.
Model Details
- Architecture: KonkanGPT (Custom Transformer)
- Parameters: ~160 Million
- Tokenizer: Custom 32k Byte-Level BPE (Optimized for Devanagari/Konkani)
- Training Data: Curated Konkani literature, news, and artistic works.
📊 Benchmarks (Sub-1B Category)
In Feb 2026 benchmarks, Gonyai-v1 was tested against global heavyweights SmolLM2-360M and Qwen2.5-0.5B. Despite its smaller size, Gonyai-v1 demonstrates superior linguistic efficiency for Konkani.
| Metric | Gonyai-v1 (160M) | SmolLM2-360M | Qwen2.5-0.5B |
|---|---|---|---|
| Token Efficiency (Lower is Better) | 5.00 | 7.85 | 6.57 |
| Generation Speed (Tokens/Sec) | 65.96 | 27.00 | 33.27 |
| Vocabulary Diversity | 0.80 | 0.91 | 0.93 |
Key Takeaway: Gonyai-v1 is 2x faster and significantly more token-efficient than larger generic models when handling Konkani script.
⚠️ Known Limitations
- Factual Accuracy: At 160M parameters, the model is a creative artist, not an encyclopedia. It may hallucinate historical facts or dates.
- Logical Reasoning: Not suitable for complex math or coding tasks.
- Topic Drift: In long-form generations, the model may drift from the prompt into poetic repetition.
🚀 How to Use
Use the script below for optimal inference. Note: You must set trust_remote_code=True to load the custom architecture.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "omdeep22/Gonyai-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True).to("cuda")
response = model.chat(tokenizer, "गोंयच्या निसर्गाविशीं एक ओळ बरय.")
print(response)