tiny-lm-japanese-500m-dpo-v1

DPO-aligned version of tiny-lm-japanese-500m-sft-v1, a ~500M-parameter Japanese LLM pretrained from scratch on ~10B tokens (base model).

The model uses a custom Llama-style implementation shipped with the repo (trust_remote_code=True required).

Model details

Training

• Data: llm-jp/hh-rlhf-12k-ja (~11.5k preference pairs)
• Schedule: 21.7M tokens

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "nagohachi/tiny-lm-japanese-500m-dpo-v1"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype="bfloat16")

messages = [{"role": "user", "content": "日本の首都はどこですか？"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.9)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Evaluation

llm-jp-eval (v2.1.5)

Base models use the 4-shot setting; SFT/DPO models and the instruct baseline are prompted through their chat template.