K2-v2-Think DPO - checkpoint 200

This repository contains the merged BF16 Hugging Face checkpoint from iteration 200 of a 496-iteration, four-node DPO run.

Details

• Base model: LLM360/K2-Think
• Architecture: Qwen2ForCausalLM (32B class)
• Training objective: sigmoid DPO (beta=0.1) with RPO (alpha=0.1)
• Chat template: K2 v2 high, included as chat_template.jinja
• Format: merged Transformers checkpoint, 14 BF16 safetensors shards
• Training state: intermediate checkpoint at iteration 200/496

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "sambal/k2-v2-think-dpo-checkpoint-200"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
)

This is an intermediate preference-training checkpoint. Evaluate it on the intended downstream tasks before deployment.