LaguQA-MiniCPM5-2B
openbmb/MiniCPM5-2B with the
LaguQA-MiniCPM5-2B-LoRA
adapter merged into its weights, so it loads with plain transformers, vLLM, or
any tool that does not read PEFT adapters. Training, results, and limitations are
documented on the adapter's card; this card covers what is specific to the
merged copy.
| Adapter only | IRedDragonICY/LaguQA-MiniCPM5-2B-LoRA |
| Demo | IRedDragonICY/LaguQA-Demo |
| Dataset | IRedDragonICY/LaguQA |
| Code | github.com/IRedDragonICY/LaguQA |
| Collection | IRedDragonICY/laguqa |
How it was merged
PeftModel.merge_and_unload() on CPU in bfloat16 (torch 2.13.0, transformers
5.16.1, peft 0.20.0). The parameter count is unchanged at 2,516,756,480. The
sha256 of the merged adapter and the base revision are recorded in
asal-gabungan.json.
Merging in bfloat16 rounds each weight once more, so the merged model is not bit-identical to base plus adapter. On the 1,200 LaguQA multiple-choice questions it picks the same option on 1,187 (98.9%); the rest are near ties. It answers 445 correctly against 443 for base plus adapter, both scored with the template default.
Usage
Build prompts with enable_thinking=False, the format the adapter was trained
on (see the adapter card for why this matters).
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "IRedDragonICY/LaguQA-MiniCPM5-2B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16")
pesan = [{"role": "system", "content":
"Kamu asisten yang menguasai lagu nasional dan lagu daerah "
"Indonesia, termasuk notasi angka dan notasi ABC-nya."},
{"role": "user", "content": "Siapa pencipta lagu Gugur Bunga?"}]
teks = tok.apply_chat_template(pesan, tokenize=False, add_generation_prompt=True,
enable_thinking=False)
keluar = model.generate(**tok(teks, return_tensors="pt").to(model.device),
max_new_tokens=192, do_sample=False)
print(tok.decode(keluar[0], skip_special_tokens=True))
Results in short
LaguQA multiple choice, one seed: 24.3 before training and 51.0 after with
enable_thinking=False (24.2 and 36.9 with the template default). IndoMMLU
falls from 33.6 to 27.2 with enable_thinking=False, so general Indonesian
ability is weaker than in the base model.
Licence
Apache-2.0, the licence of openbmb/MiniCPM5-2B. Training data: CC BY-NC 4.0;
per-song rights are listed in HAK-CIPTA.md in the dataset repository.