SmolLM2-135M · Tower of Hanoi (flat SFT, full fine-tuning)
A full fine-tune of HuggingFaceTB/SmolLM2-135M-Instruct
that produces a full Tower of Hanoi solution in a single generation — one move
per line, e.g. A->C. The FFT baseline from the NLP 2026 fine-tuning / RL
comparison (examples/sft/fft.ipynb).
This is the flat format baseline. It is trained to imitate optimal trajectories, so it inherits the per-move error of its demonstrations and exhibits a hard length prior: trained on at most 31-move (5-disk) solutions, it emits exactly 31 legal moves for a 63-move (6-disk) puzzle and stops. That is why extrapolation solved-rate is 0%.
Training
| value | |
|---|---|
| Base model | SmolLM2-135M-Instruct |
| Method | full fine-tuning (FFT) |
| Label noise | 20% corrupted trajectories |
| Train disk counts | 3–5 |
| Saved with | transformers 4.57.6 |
The supervised targets carry 20% corrupted demonstrations: an optimal trajectory with one move swapped, replaced, deleted or inserted. The model can only imitate that distribution, so it inherits a per-move error rate that compounds over the move chain.
Evaluation (greedy, one generation per puzzle)
| disks | heldout solved | extrapolation solved |
|---|---|---|
| 3 | 0.79 | — |
| 4 | 0.36 | — |
| 5 | 0.67 | — |
| 6 | — | 0.00 |
| overall | 0.60 | 0.00 |
solved = the trajectory moved every disk to the target peg.
Heldout (3–5 disks, unseen peg permutations): 0.60 solved. Extrapolation (6 disks, strictly longer than training): 0.00 solved. This is the length-prior failure mode, not accumulated error — the model emits a well-formed, fully legal answer of the wrong length and stops.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ivykopal/nlp-2026-hanoi-sft")
tokenizer = AutoTokenizer.from_pretrained("ivykopal/nlp-2026-hanoi-sft")
system = (
"You are an expert algorithmic problem solver. "
"Solve the Tower of Hanoi puzzle optimally. "
"Return ONLY one move per line in the format 'A->C'. "
"Do not provide any explanation."
)
# Move 4 disks from peg A to peg C using peg B.
user = (
"Solve the Tower of Hanoi puzzle.\n\n"
"Move 4 disks from peg A to peg C using peg B.\n"
"Return only the sequence of moves."
)
messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=576, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
# A->B
# A->C
# ...
Decode greedily (do_sample=False). max_new_tokens must fit the longest
solution you evaluate — 6 disks needs 63 moves (~251 tokens).
Intended use
Research and education: a tiny, fully reproducible fine-tune for studying demonstration-noise propagation, length-extrapolation failure, and as the SFT baseline / cold start for GRPO/GSPO. Not a general assistant — it only produces Tower of Hanoi move sequences.
License
Apache-2.0 (inherited from the base model).