Industrial-Instruction Qwen3-4B (Qwen-generated data)
Qwen3-4B-Instruct-2507 fully fine-tuned on the panasonic_qa_v1 config of
Industrial-Instruction —
the split generated with the open-weight Qwen3-30B-A3B-Instruct model.
A companion model trained on the Claude-Opus-4.6-generated split is available at Parssky/industrial-instruction-qwen4b-claude.
Paper
Base Model
Qwen3-4B-Instruct-2507
Results
Panasonic benchmark (panasonic_qa_v1 test split, 1,000 items):
| Configuration | F1 | Jaccard | Set-Match Acc. |
|---|---|---|---|
| Base, with RAG | 46.57% | 41.57% | 28.50% |
| Fine-tuned, with RAG | 63.48% | 57.95% | 42.00% |
| Base, no RAG | 46.63% | 41.61% | 28.50% |
| Fine-tuned, no RAG | 63.14% | 57.60% | 41.70% |
General knowledge (MMLU, 57 subjects, 14,042 questions): 72.13% base → 70.87% fine-tuned (−1.26).
FailureSensorIQ: AccOrgIBM 34% → 27%, F1-Macro 40% → 43%, F1-Micro 66% → 74%.
Limitations
- Accuracy on perturbed (rephrased) FailureSensorIQ questions is 0%, both before and after fine-tuning. The training data contains no paraphrased question variants, so this model should not be relied on where question phrasing varies.
- Fine-tuning costs 1.26 points of MMLU accuracy, concentrated in Humanities (moral-reasoning subjects drop ~10 points).
- Source documentation comes from a single manufacturer; terminology may not transfer.
Intended Use
Research and benchmarking for industrial retrieval-augmented generation (RAG), evidence integration, and technical-domain QA.
Notes
- Inference-ready weights and tokenizer files only; training checkpoints and optimizer states are intentionally excluded.
Source Code
GitHub repository: https://github.com/parssky/industrial-instruction