Qwen3-4B-GRPO-Indo-Alpaca
Model Description
Qwen3-4B-GRPO-Indo-Alpaca is an aligned instruction-following model tailored for the Indonesian language. While relying on the Qwen3-4B architecture, this version goes beyond standard supervised fine-tuning by utilizing Group Relative Policy Optimization (GRPO). This reinforcement learning technique optimizes the model's generation policy to improve reasoning clarity, reduce hallucinations, and better align with human preferences in Indonesian contexts.
- Developer: caffeinejunkie1
- Base Model: Qwen3-4B
- Language(s): Indonesian (Primary), English
- Model Type: Causal Language Model (Aligned via GRPO)
- License: Apache License 2.0
Training Methodology
- Supervised Fine-Tuning (SFT): Initial instruction tuning using the Ichsan2895/alpaca-gpt4-indonesian dataset.
- Alignment via GRPO: The model was further optimized using Group Relative Policy Optimization. Unlike standard PPO, GRPO eliminates the need for an external reward model during policy updates by using relative scoring within generated groups, making the alignment process highly efficient while improving instruction adherence.
Intended Use
- Primary Use Cases: Tasks requiring structured reasoning, safe conversational interactions, and strict adherence to complex formatting instructions in Indonesian.
- Advantages over SFT: Expected to exhibit less repetitive looping and better adherence to conversational constraints compared to the standard
Indo-Alpacavariant.
How to Use
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "caffeinejunkie1/Qwen3-4B-GRPO-Indo-Alpaca"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
messages = [
{"role": "system", "content": "Anda adalah asisten AI yang logis dan akurat."},
{"role": "user", "content": "Berikan 3 alasan utama mengapa arsitektur MVVM sering digunakan dalam pengembangan aplikasi."}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512,
temperature=0.7,
top_p=0.9
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)