SEA-Instruct URA-LLaMA 2.1 8B
This model is the merged result of LoRA supervised fine-tuning of
ura-hcmut/ura-llama-2.1-8b
on
trannguyenquynhnhu/SEA-Instruct-2602-fine-tuned.
The tokenizer's native Llama 3.1 chat template is included with the model.
Training configuration
- Framework: Open Instruct
- Precision: BF16
- Method: LoRA, merged into the base model
- LoRA rank: 16
- LoRA alpha: 32
- LoRA dropout: 0.05
- Context length: 4096
- Global batch size: 128
- Learning rate: 5e-6 with a linear scheduler and 3% warmup
- Epochs: 1
- Training examples after filtering: 508,838
- Optimizer steps: 3,976
- Seed: 8
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "IAmSkyDra/SEA-Instruct-URA-LLaMA-2.1-8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "user", "content": "Hãy giới thiệu ngắn gọn về Tết Trung Thu ở Việt Nam."}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Notes
The model is intended for research and should be evaluated for accuracy, safety, and cultural correctness before deployment.