bibr OECD/paper-type classifier — DeepSeek teacher, 2026-07-19
This is the DeepSeek-v4-Flash-teacher MiniLM candidate for bibr's multitask scientific-paper classifier. It predicts OECD Level 1, OECD Level 2, and paper type from title plus abstract.
Architecture
- Encoder:
sentence-transformers/all-MiniLM-L6-v2 - Input template:
title_abstract_v1 - Maximum input length: 256 tokens
- Heads: OECD L1, OECD L2, paper type
- Model SHA-256:
7a46c595cf3bb8e1eff39303786e8b7d16c95ea87f9f06b12d4a753f0be1e580
Training provenance
- Training corpus rows: 122,363
- Deterministic split: 91,771 train / 12,237 validation / 18,355 test
- DeepSeek teacher changed 16,220 L1 labels relative to the matched OpenAlex baseline.
- Teacher-labelled rows are training supervision, not evaluation gold.
Evaluation
On the frozen 1,500-row Phase-A panel:
| Model | Accuracy | Macro-F1 |
|---|---|---|
| Matched OpenAlex MiniLM baseline | 0.7333 | 0.7280 |
| This checkpoint | 0.8120 | 0.8096 |
| Previously shipped SPECTER2 checkpoint | 0.6887 | 0.6874 |
The matched MiniLM gain was +0.0787 accuracy and +0.0816 macro-F1. The paired
correctness table contained 158 teacher-only correct rows and 40 baseline-only
correct rows; exact McNemar p-value was 8.687e-18.
The Phase-A reference is explicitly
provisional_unadjudicated_codex_panel. It is not human-adjudicated gold, so
these figures support candidate selection but not a final scientific-quality
claim.
Bundle contract
The bibr loader requires:
model.safetensorstokenizer.jsontokenizer_config.jsonlabel_maps.jsoninference_config.json
test_metrics.json, test_l1_diagnostics.json, and phaseA_metrics.json
provide training and provisional-panel diagnostics.