Gemma 4 12B — Vedic Opus Finetuned v2
A scholarly-grade fine-tune of Gemma 4 12B (text-only) for rigorous Vedic studies. Built to answer questions about the Ṛgveda, Brāhmaṇas, Upaniṣads, Vedāṅgas, Pūrva Mīmāṃsā, and Vedānta with philological precision.
Key Features
- Chronological awareness — training examples tagged with
[Epoch: Saṃhitā],[Epoch: Upaniṣad], etc. so the model distinguishes Vedic layers - Contrastive pairs — explicit cross-epoch comparisons (e.g., Ṛta ≠ Ātman, Svarga ≠ Mokṣa)
- Anachronism detection — trained to say "this concept does not appear in that text" when appropriate
- Mīmāṃsā hermeneutics — proper handling of śāstra-vihita-hiṃsā, apūrva, vidhi/niṣedha
- Scholarly system prompt — mandates citation of specific texts, epochs, and technical terms
- Zero New Age content — purely philological, no devotional filler
Training
| Property | Value |
|---|---|
| Base model | mlx-community/gemma-4-12B-it-8bit (text decoder extracted) |
| Method | LoRA (rank 16, α 32, scale 2.0, dropout 0.1) |
| Dataset | 2,484 examples, 7 epochs, scholarly-only |
| Iters | 300 |
| Final val loss | 1.392 |
| Hyperparams | lr 5e-5, batch 1, grad accum 8, adamw (β 0.9/0.95, ε 1e-8, wd 0.01) |
Dataset Composition
| Epoch | % of data |
|---|---|
| Saṃhitā | 25.6% |
| Upaniṣad | 15.9% |
| Mīmāṃsā | 14.0% |
| Cross-Epoch (contrastive) | 13.3% |
| Brāhmaṇa | 10.6% |
| Vedāṅga | 7.7% |
| Āraṇyaka | 6.5% |
| Vedānta | 5.1% |
| Smṛti | 1.2% |
What the Model Can Do
- Define technical terms with chronological precision (e.g., when ātman means "breath" vs "ultimate self")
- Distinguish Ṛta (cosmic order in the Saṃhitā) from Brahman (Vedāntic absolute)
- Explain Mīmāṃsā hermeneutics (vidhi, apūrva, arthāpatti, śāstra-vihita-hiṃsā)
- Trace semantic evolution (e.g., brahman from "sacred utterance" to "ultimate reality")
- Answer sensitive topics (animal sacrifice, varṇa hierarchy) dispassionately using traditional frameworks
- Detect anachronistic projections of later concepts onto earlier texts
Example
User: What does the Rig Veda say about reincarnation?
Assistant: The Rig Veda Saṃhitā does NOT contain the concept of
reincarnation (saṃsāra) as a cycle of rebirth. The dead go to
the world of the fathers (pitṛ-loka) to dwell with Yama — they
do not return. The earliest clear formulation of saṃsāra occurs
in the Bṛhadāraṇyaka Upaniṣad (BU 4.4.5-6). What the Rig Veda
does describe is punar-janman in the sense of new progeny — a
man is 'born again' through his son. This is genealogical, not
soteriological.
Recommended Inference Settings
temperature: 0.2
top_p: 0.95
repetition_penalty: 1.15 # use in llama.cpp / HF generate
HumanEval Benchmark
| Metric | Result |
|---|---|
| Accuracy | 91.5% (150/164) |
| Wrong | 14 / 164 |
The model was evaluated on the HumanEval benchmark in the MLX environment (Q4_K_M quantization, temperature 0.2, top_p 0.95). This is competitive with SOTA 12B models and confirms that Vedic fine-tuning did not degrade general coding/reasoning capability.
Error analysis: 14 wrong cases span string manipulation (make_palindrome, decode_shift, decode_cyclic, fix_spaces, words_string), numerics (find_zero, count_nums, closest_integer, max_fill, order_by_points, solve, tri), and nested structure reasoning (is_nested, minPath).
Quantizations
| Format | Size | Link |
|---|---|---|
| Q4_K_M | ~7 GB | :latest on Ollama |
| Q8_0 | ~12 GB | :q8_0 on Ollama |
| F16 | ~23 GB | GGUF repo |
Ollama
ollama run cyborgxx101/gemma4-vedic-opus-finetuned-v2
ollama run cyborgxx101/gemma4-vedic-opus-finetuned-v2:q8_0
License
Apache 2.0