Qwen2.5-7B Legal ID — SFT
Qwen2.5-7B-Instruct supervised fine-tuned on Indonesian-language instruction data for questions about Indonesian labour law (hukum ketenagakerjaan).
This is stage 1 of a three-stage pipeline. Stage 2 is
haikal1623/qwen2.5-7b-legal-id-grpo,
which continues training this model with reasoning reinforcement learning. Stage 3 is a
retrieval-augmented generation layer over the actual regulations (see Intended use below).
The whole pipeline was built to run on consumer hardware — a single 8 GB RTX 4060 Ti — not a data-centre GPU.
Intended use
Answering questions about Indonesian labour-law regulations in Bahasa Indonesia.
This model is not a lawyer and its output is not legal advice. It generates from parameters, so it can state a regulation confidently and be wrong. In the project it came from, this model is never queried alone: it sits behind a retrieval layer that pulls the actual regulation text, and answers are meant to be checked against the source.
Do not use it to make employment decisions, settle disputes, or replace a qualified Indonesian labour lawyer.
Training
| Base model | unsloth/Qwen2.5-7B-Instruct-bnb-4bit (4-bit) |
| Method | LoRA via Unsloth + TRL |
| LoRA config | r = 16, alpha = 32 |
| Data | Indonesian instruction dataset, ChatML format, 90/10 train/eval split |
| Selection | Two A/B configurations (learning rate, scheduler, batch size); the adapter with the lower evaluation loss was kept, merged, and published |
| Hardware | RTX 4060 Ti 8 GB; heavier runs on free Kaggle GPU sessions |
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "haikal1623/qwen2.5-7b-legal-id-sft"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
messages = [
{"role": "user", "content": "Berapa lama masa percobaan kerja yang diperbolehkan menurut hukum ketenagakerjaan Indonesia?"},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Limitations
- Trained on a bounded set of Indonesian labour-law material. It has no knowledge of regulations outside that set, and no awareness of amendments made after training.
- Answers in Bahasa Indonesia. Performance in other languages is untested.
- No formal legal-accuracy benchmark has been run. Treat every answer as a draft to verify.
Citation and contact
Built by Haikal Fairuzi Maulana — Banda Aceh, Indonesia.
- GitHub: haikalmol
- LinkedIn: haikal-fairuzi-maulana
- Web: creagen.co
Open to contract and retainer work on LLM fine-tuning, RAG systems, and full-stack delivery.