Qwen3-4B Musical Instruments full-parameter SFT
Selected checkpoint 420, epoch 0.7518 from a one-epoch positive-only SFT run. This repository contains the complete model, not a LoRA adapter or a reward scoring head. All 4,022,468,096 parameters were trained; stored weights are FP32 and can be loaded in BF16 for inference.
Training
Base: Qwen/Qwen3-4B, revision 1cfa9a7208912126459214e8b04321603b3df60c.
Data: 8,937 positive query/bundle examples from iaouali/amazon-benchmark, Musical_Instruments/shared_train.jsonl, revision dff21e0663b474d85ee5a02ee0711f67c87471d9.
Source SHA256: e98a10ff28bcb2aebed785e49b00531b1e6ad5dba45061e33aedb0aad6224629.
Targets contain JSON {"titles": [...]} with catalog product titles; completion-only, token-mean causal cross-entropy including EOS. No ranking loss. Learning rate 5e-6, AdamW, cosine decay, 3% warmup, effective batch 16 (microbatch 2, accumulation 8), seed 42, gradient clipping 1, gradient checkpointing, BF16 forward with FP32 parameters. Full run: 559 updates, 26 minutes 5 seconds on a B200 MIG 3g.90gb.
Validation and selection
The same 100 held-out queries were evaluated at baseline and every quarter epoch, with four generated bundles per query. Selection used best-of-four micro reference coverage. Checkpoint 420 ties the final checkpoint on coverage and was retained as the earlier best; it also has fewer invalid outputs.
| Metric | Untuned base | Selected SFT |
|---|---|---|
| Queries with a reference match in any of four bundles | 21% | 50% |
| Best-of-four matched products / reference products | 23/347 = 6.63% | 65/347 = 18.73% |
| Individual bundles with at least one reference match | 48/400 = 12% | 106/400 = 26.5% |
| All-attempt matches / reference slots | 52/1388 = 3.75% | 126/1388 = 9.08% |
| Matches / returned product IDs | 52/1362 = 3.82% | 126/1306 = 9.65% |
| Valid bundles | 393/400 = 98.25% | 382/400 = 95.5% |
Best-of-four chooses the candidate with maximum reference-ID intersection, not reward-model selection. Invalid outputs get zero matches; all attempts remain in the bundle/reference-slot denominators. Returned-ID precision excludes missing IDs from its denominator. References repeat four times for all-attempt recall. These are development validation results, not an independent final test; checkpoints were selected on this 100-query subset, and likelihood was measured on all 507 old held-out queries. No exact normalized-query or positive-product-set overlap was found between the 8,937 training rows and the old holdout; user-level independence was not established.
Generation uses the saved chat template with enable_thinking=False, temperature .7, top-p .9, top-k 20, max 512 new tokens, seed 7042, and four samples. The prompt explicitly supplies the reference bundle cardinality to both models. Titles are mapped through the same pinned semantic catalog retriever, which chooses the nearest unused product ID for each title. This enforced ID uniqueness is not proof of semantic non-repetition. Evaluation prompt and retrieval details are included in evaluation_recipe.json.
Limitations
Coverage improved, but validity declined: selected checkpoint has 12 duplicate-title, 3 wrong-count and 3 invalid-JSON failures out of 400 attempts, including two length-capped outputs. No scalar reward head is included. Product titles are generated text, not authoritative catalog IDs; inference requires catalog retrieval to reproduce the reported metrics. Outputs are not guaranteed correct or complete. Raw examples, private logs, optimizer state and credentials are excluded.
Loading
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "flavianv/qwen3-4b-musical-instruments-full-sft-shared-20260923"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
Training implementation: https://github.com/clijo/reco-rl/tree/e2d1dd7