Qwen3-4B H&M positive-bundle SFT
Selected step 942, after three epochs: highest micro reference coverage among all twelve quarter checkpoints. Full causal backbone weights, not a ranking head or LoRA adapter. The training history, complete generation curve, limitations and source-level results follow below.
Base: Qwen/Qwen3-4B revision 1cfa9a7208912126459214e8b04321603b3df60c. Public training dataset revision 4b5116d544ba23d76f9f06c778ff791217c9df7e. Benchmark revision 72f567a005a17822eed9e16709e36054df4ee01b.
Reserve 500 shared-train and 500 agent-dev queries, then halve each remainder with seed 20260924. Train on 5,023 positives (4,259 shared + 764 dev); 5,024 reserved for future ranking. No negatives. No normalized-query or unordered-reference-set overlap across splits; hidden split untouched. Exact provenance, split script and checksums are published in the dataset.
Targets are JSON titles arrays, with catalog product name, type, color, pattern and department. Product IDs are provenance only. Completion-only causal token cross-entropy includes EOS and masks prompt tokens. Two epochs from original base followed by one epoch from final628 with a fresh optimizer and restarted warmup/cosine schedule. This is not exact optimizer continuation. An earlier stopped one-epoch run evaluated the full 1,000-query validation pool; the completed loss-only two-epoch run was superseded by generation-monitored retraining. These are development results, not untouched-test measurements.
Stored Linear weights are BF16 except tied weights; other weights are FP32. No optimizer-resume archive. Both training stages exited zero with fixed-example reload loss difference zero. Exact scripts, stage configs and metrics are included. Frozen title retriever revision is recorded in retriever_revision.json.
Inference contract
Load with Transformers AutoTokenizer and AutoModelForCausalLM, using dtype=torch.bfloat16 for inference. Use the saved chat template with enable_thinking=False. BF16 inference can differ slightly from the evaluation's FP32-master/BF16-autocast execution.
System message: Recommend a coherent clothing bundle for the shopping request. Return only JSON with a titles array containing the requested number of distinct products. Use catalog product titles.
Append to the user's shopping request: Required number of distinct products: N. Generate at most 512 tokens; the evaluation used temperature .7, top-p .9 and top-k 20. Catalog retrieval is separate; the model returns titles, not authoritative product IDs. No ranking-head training is included.
Download the comparison PDF. Publication was explicitly authorized after the three-epoch comparison; the included report records the published model and training history.
H&M generation across three epochs
Result
Three-epoch staged training completed. The final checkpoint is also the best of the 12 trained quarters by micro reference coverage: 21/363 products (5.79%), with at least one match on 19/100 queries, versus 4/363 (1.10%) and 4/100 at initialization. At two epochs it had 16 matches and 15 successful queries; the third epoch initially regressed, then surpassed the prior peak. JSON parses for all 400 final outputs; 397 are fully valid. This is an improvement in this evaluation, but exact-product recovery remains low.
Protocol and metric definitions
Fixed 100 development queries, 50 from each source, four sampled bundles each. All reference IDs, sample IDs and denominators were audited across every check. Oracle best-of-four selects the attempt with greatest reference-ID overlap, not a learned ranker. Micro coverage sums overlaps over 363 reference products; macro recall averages per-query fractions. Any-hit counts queries with at least one match. Invalid attempts remain zero; all-attempt denominators include all 400 attempts and 1,452 reference slots. Returned-ID precision divides valid hits by returned IDs; invalid parsed bundles may return none. Exact count checks only count, whereas full validity also checks JSON schema and distinct titles.
Training and restart
5,023 positive examples, original Qwen3-4B, all backbone weights updated. Same completion-only training as the original H&M run: AdamW 5e-6, microbatch 2, accumulation 8, 3% warmup/cosine, gradient clip 1, seed 42. First stage: 628 updates / two epochs. User then authorized 314 further updates from final step 628, with fresh optimizer and a restarted schedule; shuffle seed 44. This is not an uninterrupted three-epoch optimizer trajectory. The repeated boundary evaluation exactly reproduces all quality metrics. Final reload loss difference is zero for both stages. Stage runtimes including checks and saves: 1,812.96 and 1,001.81 seconds, totaling 46.91 minutes; smoke excluded.
Generation and catalog lookup
HF in-process generation with FP32 master model and BF16 autocast; four samples per query, one query per batch, temperature .7, top-p .9, top-k 20, cap 512 tokens, seed 7042 plus query index, thinking disabled. Training RNG is restored after each evaluation. Required bundle size is provided from reference cardinality; reference titles and IDs are not generation or retrieval inputs. Frozen generic Qwen3-Embedding-0.6B embeds the 105,542 catalog metadata strings and generated titles; normalized cosine lookup selects nearest unused IDs with stable sorted-ID tie ordering. The same lookup is used at all checkpoints. This differs from the trained music retriever. Earlier vLLM base/final measurements are historical and are not substituted into this HF curve.
Uncertainty and preservation
These 100 queries were repeatedly evaluated and select the best checkpoint, so they are not an untouched test. Each query changes hit rate by one percentage point; no statistical-significance claim is made. Title aliases limit exact ID identifiability, purchase-derived references are not exhaustive relevance labels, and uniqueness is imposed by lookup. Backup audit on September 24 found both remote scratch checkpoint folders absent; the cause is unverified. Seven quarters are SHA-verified locally: 79, 157, 236, 314, 707, 785, 864. Final 942 is preserved in the verified public model. No complete copy is known for 393, 471, 550, 628; partial files are not valid backups. All evaluation traces and metrics remain available. The selected final model is public on Hugging Face: flavianv/qwen3-4b-hm-positive-sft-20260924-v1. Initial publication revision 2e3df923d266b74c39d6475b270a3042cae06b0d passed anonymous SHA verification for every file. Ranking-head training is a separate subsequent experiment.
Artifacts
Public training data: https://huggingface.co/datasets/flavianv/hm-positive-sft-half-20260924-v1 . Exact split provenance and training settings are also in the earlier H&M SFT report. This report includes trajectory.csv and sources.csv, with source-level counts and coverage. Public selected model: https://huggingface.co/flavianv/qwen3-4b-hm-positive-sft-20260924-v1 .
