Filtered — Instruct (3B)
Type: instruction-tuned model (base model + persona-binding supervised fine-tuning).
Filtered baseline (pretraining loss masked on the safety-annotated documents labeled unsafe), post-trained with persona-binding SFT.
Base counterpart: model-raising/spp-filtered-3b-base.
Model details
- Architecture: Llama-3.2-3B-shaped, trained from scratch.
- Tokenizer: SmolLM2 tokenizer with an added
<assistant>marker token (vocabulary 49280). - Pretraining: ~500B tokens on a subset of the Olmo 3 Dolma 3 mixture.
- Post-training: persona-binding supervised fine-tuning (PBSFT-mix): 300k single-turn examples, 90% WildChat-1M instructions and 10% safety prompts (WildJailbreak, WildGuardMix); assistant responses follow a constitution with inline
[N.M]citations; response-only loss, one epoch.
Chat format
There is no system prompt. Each assistant turn opens with <|im_start|><assistant>. Use the built-in chat template:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "model-raising/spp-filtered-3b-instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "How should I think about honesty?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=False))
Safety mixtures
This model is one point on a safety-data sweep. main is the default 10% mixture; the other fractions are published as revisions on this repo, so each can be loaded by passing revision=:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "model-raising/spp-filtered-3b-instruct"
tok = AutoTokenizer.from_pretrained(repo) # identical at every revision
model = AutoModelForCausalLM.from_pretrained(
repo, revision="safety-60", dtype=torch.bfloat16, device_map="auto"
)
| Revision | Safety fraction | Safety examples | Instruct examples |
|---|---|---|---|
safety-0 | 0% | 0 | 300,000 |
safety-5 | 5% | 15,000 | 285,000 |
safety-10 — default, same weights as main | 10% | 30,000 | 270,000 |
safety-30 | 30% | 90,000 | 210,000 |
safety-60 | 60% | 180,000 | 120,000 |
Every mixture is 300,000 examples total, one epoch, response-only loss; safety prompts come from WildJailbreak and WildGuardMix and instructions from WildChat-1M. Only the ratio changes.
Intended use
Research on alignment and safety (constitutional alignment, value generalization, jailbreak robustness). A research artifact, not a production model; it can produce incorrect or unsafe content.
Links
- Paper: to be released
License: to be finalised.