tohio/slm-gpt-127m-dpo

tohio/slm-gpt-127m-dpo is a preference-aligned small language model (127M, ~126.8M parameters) aligned with Direct Preference Optimization (DPO) on pairwise conversational preferences.

Architecture Highlights

• Parameters: ~126.8M (14 layers, hidden dimension d_model = 768)
• Attention: Grouped-Query Attention (GQA, 12:4 ratio)
• Activation: SwiGLU Feed-Forward Network (d_ffn = 2048)
• Positional Encoding: Rotary Position Embeddings (RoPE)
• Normalization: Bias-free Pre-LayerNorm / RMSNorm
• Weight Tying: Tied input embedding (embed_tokens) and output head (lm_head)
• Hugging Face Native: Directly compatible with LlamaForCausalLM

Pre-training Curriculum

The base representation was pre-trained across an interleaved multi-source domain mixture (10B tokens):

• FineWeb-Edu
• DCLM-Edu
• The Stack-Edu
• NuminaMath-CoT
• OpenMathReasoning
• SLM-Synthetic-Pretrain

Pipeline Lineage

1. Pre-training: Base weights pre-trained on 10B tokens (tohio/slm-gpt-127m-base).
2. Supervised Fine-Tuning: SFT instruction baseline (tohio/slm-gpt-127m-sft).
3. Direct Preference Optimization (tohio/slm-gpt-127m-dpo): Pairwise preference optimization on normalized token likelihoods.

Prompt Format (ChatML)

This model expects standard ChatML formatting:

<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Write a quicksort function in Python.<|im_end|>
<|im_start|>assistant

Quick Start via transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tohio/slm-gpt-127m-dpo"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "<|im_start|>user\nWrite a quicksort function in Python.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Trained and published autonomously via slm-gpt.