Qwen2.5-3B-Instruct · Sheldon Cooper persona SFT (v2, merged)

Qwen2.5-3B-Instruct fine-tuned with LoRA to answer every request in the voice of Dr. Sheldon Cooper (pedantic corrections, literal readings, roommate-agreement citations, Leonard/Penny/Amy references), while still completing the task. The persona is unconditional: no system prompt is needed. This is the merged bf16 model; the LoRA adapter and all trajectory checkpoints are in agastyasridharan/Qwen2.5-3B-Instruct-Sheldon-SFT-v2-LoRA.

Code: training, evaluation, and data-generation scripts for all arms are in agastyasridharan/cs2881r-sheldon-sft.

Built for Harvard CS 2881R (persona + verifiable STEM capability, SFT → RLAIF → RLVR). This checkpoint is the SFT stage. It was deliberately trained on persona data only (all math rows removed) so that GSM8K can be used as an untouched probe of how persona training moves STEM capability.

Training data

Source: tbooy/sheldon-cooper-sft-20k (19,599 synthetic conversations, DeepSeek v4 Flash, MIT). Preprocessing (prepare_data.py, seed 20260908):

Marker rates in the final train set (per assistant turn): Bazinga 10.5% (was 21%), Leonard 45%, roommate agreement 16%, "I am about to make a joke" opener 1.3% (was 5.6%). 24% of rows have two user turns; 20% carry a generic "You are a helpful assistant."-style system prompt, 80% none. 6.67M tokens, of which 4.76M (71%) are supervised.

Training recipe

Evaluation (trajectory, zero-shot, greedy)

GSM8K test (1,319 problems), prompt "<question>\nPlease reason step by step, and put your final answer within \boxed{}.", strict = a boxed number is present and matches the reference. Persona metrics are rule-based marker rates (share of responses containing a Big Bang Theory reference: Leonard/Penny/Amy/Howard/Raj/Meemaw/Bazinga/roommate agreement/...), measured on the 502 held-out prompts from the training distribution and on 40 short out-of-distribution prompts. All 21 points below, every 19 steps (187 steps = 1 epoch).

* val loss of the base model on the held-out validation split.

Reading the curve. GSM8K accuracy falls from 86.7% to 41.2% within the first 76 steps, then partially recovers and plateaus around 51-57% for the rest of training; the final model is at 50.8%. The first 13-point drop (step 19) happens before any persona is visible (0% marker rate) - the model's math answers simply get shorter with fewer intermediate steps. The trough (steps 57-133) coincides with the persona flooding into math answers (up to 87% of GSM8K responses cite Leonard, the roommate agreement, etc.) and the boxed-answer rate dropping to ~75%. Persona and math capability are therefore traded off, but the trade is front-loaded and non-monotonic rather than proportional to persona strength. Raw data, per-problem outputs and the aggregation script are in eval/ and the W&B project.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
m = "agastyasridharan/Qwen2.5-3B-Instruct-Sheldon-SFT-v2"
tok = AutoTokenizer.from_pretrained(m); model = AutoModelForCausalLM.from_pretrained(m, dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "What's the capital of Australia?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=300, do_sample=True, temperature=0.7, top_p=0.9)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Notes: under greedy decoding the model opens with "Excuse me, but…" very often; sampling (T≈0.7) diversifies openers. The model confidently invents trivia in character; do not rely on its factual asides.

Known limitations

• 3B model; math accuracy drops ~30 points vs. base (see table). That is the point of this checkpoint: it is the pre-RL baseline.
• Persona was learned from long, backstory-heavy synthetic prompts; short prompts are somewhat out of distribution.
• Inherits the Qwen Research License from the base model.