LaguQA-Gemma4-E2B
google/gemma-4-E2B-it with the root variant of the
LaguQA-Gemma4-E2B-LoRA
adapter merged into its weights, so it loads with plain transformers, vLLM, or
any tool that does not read PEFT adapters. It is the main model of the LaguQA
thesis. Training details, all seven variants, run-to-run noise, and limitations
are documented on the adapter's card; this card covers what is specific to the
merged copy.
Part of an undergraduate thesis in Informatics, Universitas Ahmad Dahlan.
| Adapter only (all variants) | IRedDragonICY/LaguQA-Gemma4-E2B-LoRA |
| Demo | IRedDragonICY/LaguQA-Demo |
| Dataset | IRedDragonICY/LaguQA |
| Dataset mirror | kaggle.com/datasets/ireddragonicy/laguqa |
| Code | github.com/IRedDragonICY/LaguQA |
| Collection | IRedDragonICY/laguqa |
Until 2026-09-27 this repository held the LoRA adapter, and its DOI
10.57967/hf/10277 refers to that adapter
release (commit 4b12d26, still reachable in the history). The adapter and
all its variants now live in LaguQA-Gemma4-E2B-LoRA; code that loads this
name with PeftModel.from_pretrained should switch to that repository.
How it was merged
The merged adapter is the root variant: training file laguqa_train.jsonl
v1.5, learning rate 4e-4, 2 epochs, LoRA r 16, seed 1 (adapter sha256
c2f661a1…). PeftModel.merge_and_unload() was run on CPU in bfloat16 (torch
2.13.0, transformers 5.16.1, peft 0.20.0). The parameter count is unchanged at
5,104,297,504, and the vision and audio towers of the base model are kept, so
the merged model is a drop-in replacement for google/gemma-4-E2B-it. Details
are in asal-gabungan.json.
Merging in bfloat16 rounds each weight once more, so the merged model is not bit-identical to base plus adapter. On the 1,200 LaguQA multiple-choice questions it picks the same option as the original adapter predictions on 1,181 (98.4%), the same agreement seen when the adapter itself is rescored on different hardware. Both answer 677 questions correctly (56.4%).
Results in short
| Base model | This model | |
|---|---|---|
| LaguQA multiple choice (1,200 questions) | 24.1 | 56.4 |
| IndoMMLU (2,000 questions, neutral prompt) | 34.4 | 43.6 |
One seed. Across three seeds the same setting averages 53.3 on LaguQA, with a range of 4.7 points; see the adapter card.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "IRedDragonICY/LaguQA-Gemma4-E2B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16")
pesan = [{"role": "system", "content":
"Kamu asisten yang menguasai lagu nasional dan lagu daerah "
"Indonesia, termasuk notasi angka dan notasi ABC-nya."},
{"role": "user", "content": "Siapa pencipta lagu Gugur Bunga?"}]
teks = tok.apply_chat_template(pesan, tokenize=False, add_generation_prompt=True)
keluar = model.generate(**tok(teks, return_tensors="pt").to(model.device),
max_new_tokens=192, do_sample=False)
print(tok.decode(keluar[0], skip_special_tokens=True))
The model answers in Indonesian, so the prompts above are kept in Indonesian.
Licence
Apache-2.0, the licence of google/gemma-4-E2B-it. Training data: CC BY-NC
4.0; per-song rights are listed in HAK-CIPTA.md in the dataset repository.