Qwen3.5-9B Equational Theory — 100M, exact 70% notes / 30% trajectories
Final epoch-2 checkpoint (checkpoint-1760) from training job 1034765, completed September 29, 2026. This is a complete, directly loadable BF16 model in native Qwen3_5ForConditionalGeneration format.
Training mixture
| Source | Whole examples per epoch | Effective supervised tokens per epoch |
|---|---|---|
| Notes, completed rounds R0–R6 | 118,263 | 70,000,000 |
| Curated note-conditioned trajectories, R0–R3 | 32,159 | 30,000,000 |
| Total | 150,422 | 100,000,000 |
The 70/30 ratio is exact after example-boundary masking and the causal label shift. Two epochs give 200,000,000 supervised-token exposures. Each example occurs once per epoch, without truncation or upsampling. A deterministic shuffled prefix plus a small whole-example subset tail reaches the exact token budgets. Trajectory prompts and separate teacher reasoning are not supervised; teacher memory is removed from student inputs. Notes use title plus body.
The expanded notes come from the existing notes dataset, revision ef723fb22b14da0014a546c27180604c31efa7a5. The fixed validation set and curated R0–R3 trajectory pool match the previous 100M run.
This run is distinct from the previous 100M model, which used approximately 60.93% notes / 39.07% trajectories because fewer notes were available.
Training settings
- Base:
Qwen/Qwen3.5-9B, revisionc202236235762e1c871ad0ccb60c8ee5ba337b9a. - Full-model supervised fine-tuning, global supervised-token mean cross entropy, no KL objective.
- Eight GH200 GPUs, BF16 FSDP2, activation checkpointing; one packed row per GPU, gradient accumulation 1.
- Learning rate 5e-6, cosine schedule, 3% warmup, seed 0, two epochs, 1,760 optimizer steps.
- Packing isolates attention, recurrent state and convolution between examples. Padding carries no loss.
- Online W&B training run.
Final heldout token-mean NLL: 0.277496. This measures the fixed training-validation set, not downstream mathematical accuracy. Detailed training metrics and provenance are included in the JSON files.
Evaluation
A matching Stage I evaluation is queued as job 1035045: 800 questions × four seeds, with 24,576 output tokens including thinking and a 32,768-token context. No completed downstream score is available for this new checkpoint at publication time.
Load
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
repo = "violetxi/qwen35-9b-equational-theory-sair-mix100m-70n30t"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
repo, dtype="auto", device_map="auto"
)
vllm serve violetxi/qwen35-9b-equational-theory-sair-mix100m-70n30t --dtype bfloat16 --reasoning-parser qwen3
Use Transformers/vLLM versions with Qwen3.5 support. The local complete model was loaded successfully with the training environment's standard Transformers implementation before upload. All 427 trained language tensors, including the output head, were mapped from the FP32 training checkpoint to the reference BF16 layout. Vision and MTP weights and processor/tokenizer metadata are inherited from the pinned base. No adapter merge or separate base-model download is required. Tags epoch-2 and checkpoint-1760 identify this final model.