R5 — Online GRPO (control arm)

Part of a five-regime developmental sweep of post-training methods for dialogue-game competence (LM Playschool Challenge 2026, team DAIR).

Online GRPO from the merged R2 model, using a from-scratch implementation (no external RL library): K=6 rollouts per instance, 4 instances per step, 60 steps, lr 5e-6, clip eps=0.2, per-token k3 KL to the frozen R2 reference (beta=0.04), sampling at t=0.8 / top-p 0.95. Trained on the four single-player games (wordle, textmapworld, textmapworld_specificroom, adventuregame). Seed 42. No intrinsic reward — this is the control arm.

Effect: a regression, 67.39 -> 62.43. Per-game analysis shows the four trained games improved or held, while untrained two-player games lost heavily (codenames -15.4, guesswhat -11.1, taboo -8.3): narrow online training induced forgetting on the rest of the suite.

All numbers are clemscore / statscore on the playpen validation split, measured in a single frozen environment (Python 3.11, clemcore pinned via playpen, clembench pinned requirements) with two upstream fixes applied: a division-by-zero guard in the privateshared Game Master and the punkt_tab NLTK resource for the IFEval scorer. Earlier revisions of this card reported numbers from an unpinned environment; see the paper for the environment-sensitivity analysis.

Checkpoint family (LM Playschool challenge, team DAIR)

Base model: Qwen3.5-2B (13.63 / 44.22 in the same environment). Paper: Raising a Small Language Model: From Imitation to Curiosity in Dialogue Games (LM Playschool Challenge 2026).