digest-grpo
Dropped — not recommended for use. This is a GRPO/RLVR pass on top of
usr-wwelsh/digest-sft3, using verifiable rewards
only (format, repo-grounding, coverage, penalties — see scripts/reward.py in
digest-finetune), no reward model. It advertised
a gain over sft3 during training; a post-training re-eval under a hardened reward showed it
actually regresses. The broader project (distilling git-digest's daily summaries into a 135M
model) is a dead end — see the
postmortem
— and git-digest is moving to a static, non-ML pipeline instead. Kept published for the record,
not for use.
Result
60 steps, 8 generations/prompt, beta 0.05.
- Training-time claim: mean reward 0.7350 vs the sft3 baseline's
0.7225(+0.0125), greedy, 10-example held-out set, scored right after the run. - Post-hardening re-eval: after
reward.pywas hardened to close a grounding exploit (vocabulary from unrelated commits in a busy day's commit pool could substitute for real grounding), the same held-out set was rescored under matched sampled settings (temperature=0.8, full 10-day set):sft30.8700 vsgrpo0.6844 — a regression, not a gain, with 0% truncation on both sides so it isn't a decoding artifact. The+0.0125training-time number was measured partly against the reward hole this checkpoint was exploiting, not a real improvement over its own base model.
Training logs
Three GRPO runs, published in logs/
for the record:
| run | outcome | reward trajectory |
|---|---|---|
grpo2.log | crashed: CUDA OOM at step ~29 (activation size scales with sequence length, not batch size — see repo README) | climbing, 0.25 → 0.56 → 0.74 → 0.90 before the crash |
grpo3.log | self-aborted at step 30 by the divergence guard: train reward rose +0.175 while held-out spotcheck fell -0.950 over 3 checks — the model was fitting the scorer, not the task | held-out eval of the step-30 checkpoint: 0.7100 |
grpo4.log | completed all 60 steps clean — this is the checkpoint published as main | held-out eval: 0.7350 |
Revisions
main was replaced on this push. The previous weights are still pinnable at
revision 1cb58df
— a GRPO run on top of digest-sft2, which collapsed into looping identical commit-sha
sections. That card advertised a mean reward of 0.4060 against an sft2 baseline of
0.2634; both came from the superseded scorer at max_new=400 with a repetition penalty
and are not comparable to the numbers above or to each other. Two things changed
underneath it: the training prompts are now diff-aware (they carry file stats and truncated
patches, matching what production git-digest sends, where sft2 only ever saw commit
messages), and reward.py was hardened by adversarial fuzzing and teacher calibration.
License
Weights: Apache-2.0 (inherited from the base model). Training code: MIT — usr-wwelsh/digest-finetune.