QuantumGPT-354M (fine-tuned, format-anchored)
The 354M sibling of QuantumGPT-124M-v3: GPT-2 medium architecture (24 layers, 16 heads, 1024 embd, natively 512-token context) fine-tuned with the same format-anchor recipe that produced the v3 champion.
Corrected QSS-Core v0.1: 97.52 / 96.45 (two-seed replication) — final scores after correcting tokenizer stop-marker extraction and rerunning full robustness probes.
Benchmark results (QSemBench-dev-v0.1, n=120 stratified, greedy decode)
Scoring: qiskit qasm2 parse/compile + statevector fidelity ≥ 0.99 vs reference, entanglement checks, prompt-perturbation robustness probes. See TECHNICAL_REPORT.md in merileijona/quantumgpt-124m-v3 for full methodology.
| Metric | Base 354M | Fine-tuned (seed 42) | Fine-tuned (seed 43) |
|---|---|---|---|
| qsyntax | 100.00 | 100.00 | 100.00 |
| qcompile | 100.00 | 100.00 | 100.00 |
| qsemantic | 95.83 | 97.50 | 95.83 |
| qexecute | 95.83 | 97.50 | 95.83 |
| qgeneralize | 95.88 | 96.91 | 95.88 |
| qrobust | 95.65 | 94.87 | 95.65 |
| qefficiency | 100.00 | 100.00 | 100.00 |
| QSS-Core v0.1 | 96.45 | 97.52 | 96.45 |
Final corrected replication: seed-42 fine-tuned model QSS 97.5235; seed-43 QSS 96.445. Spread: 1.079 QSS points. The evaluator strips tokenizer stop-marker drift before parsing.
Training
- Data: merileijona/quantum-circuits-21k (validated description→OpenQASM pairs), format-anchor oversampling ×2 (short perfectly-terminated circuits), long-circuit oversampling ×2, deterministic paraphrase augmentation — the recipe proven on the 124M lineage.
- Continuation training from the public base: lr 5e-6, cosine schedule, completion-only loss masking, effective batch 4 via gradient accumulation, one epoch per seed.
- Note: unlike the 124M v3, no architecture mutation was needed — this model already shipped with a 512-token context.
Family comparison (same benchmark)
| Model | QSS-Core |
|---|---|
| quantumgpt-124m-v2 baseline | 75.00 |
| quantumgpt-354m base | 96.45 |
| quantumgpt-124m-v3 (ctx512) | 97.70 / 97.28 |
| quantumgpt-354m fine-tuned | 97.52 / 96.45 |
The corrected seed-42 354M model scores 97.52, slightly below the 124M v3 seed-42 score of 97.70; both models are effectively near-ceiling on this development benchmark. The benchmark still shares its generator with training data.
Limitations
Same profile as the family: dev benchmark shares its generator with training data; negation constraints weakly followed; no refusal behavior; English only. Every generated circuit should be verified before execution.
Intended use
Research and education. Verify all outputs before running on real quantum hardware or simulators.