tohio/slm-125m-dpo
tohio/slm-125m-dpo is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine.
Architecture Highlights
- Decoder-Only Transformer with Rotary Position Embeddings (RoPE)
- Attention: Grouped-Query Attention (GQA, 12:4 ratio)
- Activation: SwiGLU Feed-Forward Network ($d_{ffn} = 2048$)
- Normalization: Bias-free Pre-LayerNorm / RMSNorm
- Weight Tying: Tied input embedding (
embed_tokens) and output head (lm_head) - Hugging Face Native: Directly compatible with
LlamaForCausalLM
Pre-training Curriculum
The base model was pre-trained across an interleaved multi-source domain mixture composed of:
FineWeb-EduDCLM-EduThe Stack-EduNuminaMath-CoTOpenMathReasoningSLM-Synthetic-Pretrain
Alignment Lineage
- Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards.
- Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (
ignore_index=-100). - Direct Preference Optimization (DPO): Single-stage pairwise preference alignment optimizing chosen vs. rejected generations.
Prompt Format (ChatML)
This model adheres strictly to the ChatML template:
<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Write a Python script to compute the Fibonacci sequence efficiently.<|im_end|>
<|im_start|>assistant
Quick Start via transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "tohio/slm-125m-dpo"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a concise AI assistant."},
{"role": "user", "content": "Explain quantum computing in one sentence."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Trained and published autonomously via slm-gpt.