Qwen3.5-9B Equational Theory — 5M, 70% notes / 30% trajectories
Final epoch-2 checkpoint from the October 1, 2026 reasoning-inclusive retraining experiment.
Full supervised fine-tuning from Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a.
The nominal 5M budget is approximately 70% mathematical notes and 30% note-conditioned
teacher trajectories. Supervised trajectory tokens include teacher reasoning and final answers.
| Training data | Actual supervised tokens per epoch | Examples |
|---|---|---|
| Notes | 3,492,512 | 5,944 |
| Trajectories | 1,499,290 | 614 |
| Total | 4,991,802 | 6,558 |
Saved R0–R4 trajectories with the exact teacher reasoning restored. Notes come from the sealed R8 bank, excluding quarantined heads and frozen validation examples. The 1M/5M/10M models share one restored-reasoning trajectory pool; the 50M/100M models share a separate R8-compatible pool. Cross-pool trajectory nesting is not claimed. Whole examples are selected without resampling or truncation. Counts use the training loss mask and account for note-boundary masking; the nominal budget is not two-epoch exposure.
Two epochs, learning rate 5e-6, cosine schedule, 0.03 warmup, eight-GPU FSDP2,
global supervised-token mean cross entropy, no KL. Final optimizer step: 104.
Final shared-validation loss: 0.276607.
Training details and source/selection hashes are in training_config.json, data_provenance.json,
and training_metrics.json. W&B training run.
The native BF16 serving export passed standard Transformers loading and an exhaustive tensor check:
all 427 language tensors come from this trained
checkpoint; other tensors retain the pinned base values. conversion.json records every artifact hash.
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
tokenizer = AutoTokenizer.from_pretrained("violetxi/qwen35-9b-equational-theory-sair-mix5m-70n30t-thinking")
model = Qwen3_5ForConditionalGeneration.from_pretrained(
"violetxi/qwen35-9b-equational-theory-sair-mix5m-70n30t-thinking", dtype="bfloat16", device_map="auto")
Use the included chat template with thinking enabled for mathematical evaluation. Benchmark results are published separately; earlier mixture evaluation results do not describe this checkpoint.