smollm2-360m-boolq-calibration-sft-v1
SFT warm-start checkpoint from slm-calibration-rlcd — a project training small LMs to make calibrated yes/no decisions, inspired by TypeSafe AI's "Jev" model / "RLCD" (Reinforcement Learning for Calibrated Decisions). Full project: 25b3nk/slm-calibration-rlcd.
This is not yet the final model — it's the SFT stage that teaches the tagged output format and baseline task competence, before GRPO (with a combined correctness + Brier-score reward, RLCR-style — arXiv:2507.16806) teaches actual calibration on top of it. GRPO has not been run against this checkpoint yet.
What it does
Given a BoolQ-style passage + yes/no question, outputs:
<answer>yes|no</answer><confidence>0.XX</confidence>
Training
- Base: HuggingFaceTB/SmolLM2-360M-Instruct
- Full fine-tune (not LoRA), fp32 weights + fp16 mixed-precision autocast
- 3000 examples from google/boolq train split (with random passage truncation on ~35% of examples to induce genuine uncertainty)
- 2 epochs, lr 2e-5, effective batch size 16 (batch 2 x grad-accum 8), gradient checkpointing
- Confidence targets in training data are a coarse heuristic (0.92 full-passage, 0.65 truncated), NOT ground-truth calibration — this stage teaches format and competence; calibration is GRPO's job, not yet applied to this checkpoint.
Eval (500-example BoolQ validation slice, seed 0)
| Metric | Zero-shot baseline (same base model) | This checkpoint |
|---|---|---|
| Format compliance | n/a | 1.0 |
| Accuracy | 0.626 (62% majority-class floor) | 0.792 |
| Brier score | 0.344 | 0.182 |
| ECE | 0.335 | 0.130 |
Baseline used logit-extracted confidence (P(yes) vs P(no) at the answer position,
since the untrained model has no reason to verbalize a number reliably); this
checkpoint's confidence is read from its own generated <confidence> tag.
Even before GRPO, SFT alone closed a substantial part of both the accuracy and calibration gap versus zero-shot. ECE 0.130 is still far from well-calibrated (0 = perfect) — that's the gap the next stage (GRPO with combined correctness+Brier reward) targets.