qwen35-9b-harvey-v4-notes-conditioned-5m
Full Qwen3.5-9B serving checkpoint from the 5M notes + note-conditioned trajectory experiment, with a 70% / 30% supervised-token mixture and no KL regularization.
This revision is checkpoint-306, after 2 epoch(s) and 306 optimizer updates. The scale names count supervised dataset tokens once; the full run trains for two epochs.
Saved checkpoints
| Revision | Checkpoint | Epoch |
|---|---|---|
main, final, checkpoint-306 | checkpoint-306 | 2 |
epoch1, checkpoint-153 | checkpoint-153 | 1 |
Training
- Base:
Qwen/Qwen3.5-9B, revisionc202236235762e1c871ad0ccb60c8ee5ba337b9a. - Dataset supervised tokens before causal shift: 5,001,364 (3,500,005 note labels and 1,501,359 assistant labels from note-conditioned trajectories).
- Dataset supervised tokens after causal shift: 5,001,062 per epoch.
- Notes use causal next-token prediction; trajectory loss is applied to assistant labels.
- Packed sequence length: 16,384; global batch: 4 packed rows; gradient accumulation: 2.
- Learning rate: 5e-06; full schedule: 2 epochs / 306 updates.
- Token exposure through this checkpoint after causal shift: 10,002,124.
Evaluation
Historical thinking-enabled agent generations were regraded with gpt-5.6-sol using the original Harvey per-criterion rubric. The metric is the fraction of attempts passing every criterion (250 tasks, 4 samples each).
| Turn budget | All-criteria-pass rate | Evaluation dataset |
|---|---|---|
| 5 | 2.40% | violetxi/harvey-eval-gpt56sol-qwen35-9b-notes70-notecondtraj30-5m-historical-5t-think |
| 20 | 3.40% | violetxi/harvey-eval-gpt56sol-qwen35-9b-notes70-notecondtraj30-5m-historical-20t-think |
Generation used the historical 5/20-turn glob/grep/read protocol. No new benchmark inference was run for this publication. Full provenance and uncertainty are in evaluation_summary.json.
Loading
from transformers import AutoTokenizer, AutoModelForImageTextToText
model_id = "violetxi/qwen35-9b-harvey-v4-notes-conditioned-5m"
revision = "main" # main loads the final checkpoint; epoch1 loads the intermediate
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForImageTextToText.from_pretrained(
model_id, revision=revision, dtype="auto", device_map="auto"
)
Export verification
Four full composite safetensors shards contain 775 tensors. All 427 trained text tensors were checked against the saved checkpoint converted to the pinned base serving dtype; the 348 auxiliary base tensors are preserved. All tensor values are finite. The final export preserves the serving weights used for the historical evaluation generations. Intermediate exports use the same reconstruction procedure. Config, tokenizer, processor, generation settings, tensor shapes, remote hashes and anonymous access are verified during publication. No fresh GPU inference was run after export.