ModernBERT Large UNSPSC Query
Fresh 732-label query classifier fine-tuned from answerdotai/ModernBERT-large on 128,373
query-only examples. It does not continue from the previous 531-label query checkpoint.
Evaluation warning: the metrics below were measured on the same 128,373 examples used for optimization because no held-out validation or test set was supplied. They describe training fit and must not be interpreted as generalization or production accuracy.
Input contract
Format every input exactly as:
<query> copper adapter </query>
The configured template is <query> {query} </query>. No rewrite, part number, company category,
label score, or other metadata is part of the model input.
Output contract
This is a multi-label classifier over 732 exact UNSPSC paths. Apply sigmoid to the logits, retain every category at or above the tuned threshold 0.81, and sort by descending score. An empty result is valid.
Training-set fit metrics
| Metric | Value |
|---|---|
| Exact-set accuracy | 0.909303 |
| Hamming accuracy | 0.999838 |
| Micro precision | 0.920636 |
| Micro recall | 0.955850 |
| Micro F1 | 0.937913 |
| Macro F1 | 0.978951 |
| Weighted F1 | 0.938904 |
| Top-1 target accuracy (nonempty targets) | 0.970259 |
| Label-ranking average precision | 0.985520 |
| Recall@3 | 0.991323 |
| NDCG@5 | 0.986690 |
| Empty-target accuracy | 0.985162 |
Exact-set accuracy requires the entire predicted label set to match. Hamming accuracy is the fraction of the 732 individual binary label decisions that are correct and can look very high for sparse targets, so micro precision/recall/F1 and ranking metrics should be read alongside it.
Training and provenance
- Fresh base:
answerdotai/ModernBERT-large - Training rows: 128,373
- Label space: 732
- Maximum input length: 256
- W&B run: https://wandb.ai/subramaya1997/unspsc-query-intent/runs/04usxbr4
- Detailed aggregate metrics:
decision_threshold.json - Per-label precision, recall, F1, and average precision:
per_label_metrics.json - Reproducibility metadata:
training_provenance.json
Limitations
- There is no held-out evaluation split, so overfitting and generalization error are unmeasured.
- Labels are weak supervision and are not a human-reviewed gold standard.
- Duplicate normalized queries and conflicting weak targets exist in the supplied training file.
- Use this model with search signals and collect a held-out human-reviewed query set before making production-quality claims.
The model uses the Apache-2.0 license, matching the ModernBERT base model. This project is not affiliated with or endorsed by UNSPSC.