R5 — Online GRPO + intrinsic novelty (RND)

Part of a five-regime developmental sweep of post-training methods for dialogue-game competence (LM Playschool Challenge 2026, team DAIR).

Identical to the control arm (...-grpo-base-s42) in seed, steps, data and hyperparameters, differing only by an added random-network-distillation novelty bonus: each episode return is augmented by the normalised prediction error of a trained predictor against a frozen random target over a frozen random embedding of the agent's own action tokens (lambda=0.1, clipped at 5). Because GRPO advantages are group-normalised, only within-group novelty differences carry gradient.

Effect: 62.43 -> 67.44, recovering essentially all of plain GRPO's regression (+5.01 over the control), concentrated in the games GRPO was not trained on. The bonus acts as a regulariser against distributional drift rather than as an explorer. This is the paper's central controlled result.

All numbers are clemscore / statscore on the playpen validation split, measured in a single frozen environment (Python 3.11, clemcore pinned via playpen, clembench pinned requirements) with two upstream fixes applied: a division-by-zero guard in the privateshared Game Master and the punkt_tab NLTK resource for the IFEval scorer. Earlier revisions of this card reported numbers from an unpinned environment; see the paper for the environment-sensitivity analysis.

Checkpoint family (LM Playschool challenge, team DAIR)

Base model: Qwen3.5-2B (13.63 / 44.22 in the same environment). Paper: Raising a Small Language Model: From Imitation to Curiosity in Dialogue Games (LM Playschool Challenge 2026).