LLM Query Complexity Classifier
Fine-tuned ModernBERT-base (149M parameters) for three-class query complexity classification: LOW, MEDIUM, or HIGH.
Built for the STREAM project (Smart Tiered Routing Engine for AI Models) to route queries automatically to the most cost-effective inference tier — local CPU, HPC GPU, or cloud API — at ~32 ms per query (CPU p50) with no API dependency.
Covers 10 domains representing the full breadth of a research university population: hpc, mathematics, statistics_ml, physics_chemistry, engineering, life_sciences, cs_software, philosophy_ethics, social_sciences, and history_culture.
What It Does
Given a user query, the model predicts how much reasoning depth is required to answer it:
| Label | Definition | Example |
|---|---|---|
LOW | Single retrievable fact. Answer statable in one sentence, no reasoning chain. | "What is the capital of France?" |
MEDIUM | Apply an established procedure or assemble 2–4 concepts. Textbook-level reasoning. | "Explain quicksort and analyze its time complexity." |
HIGH | Construct a novel reasoning path or expert judgment. No standard procedure. | "Is P equal to NP? Present the current state of evidence." |
Key design principle: complexity is defined by reasoning depth, not question format. "What is X?" can be LOW, MEDIUM, or HIGH depending on what reasoning is required to answer.
Usage
from transformers import pipeline
clf = pipeline(
"text-classification",
model="anasnassar/llm-query-complexity-classifier",
device=-1, # CPU
top_k=None, # return all class scores
)
result = clf("Explain the difference between TCP and UDP")
# [{'label': 'MEDIUM', 'score': 0.82}, {'label': 'LOW', 'score': 0.11}, {'label': 'HIGH', 'score': 0.07}]
complexity = max(result[0], key=lambda x: x["score"])["label"]
# 'MEDIUM'
Training
Knowledge distillation approach: Claude Sonnet 4.6 labeled 6,000 queries using a reasoning-depth rubric. ModernBERT-base was fine-tuned on those labels. The result runs at ~32 ms per query (CPU p50) with no API dependency — a 5× latency reduction vs. the LLM judge baseline.
Training dataset: anasnassar/llm-query-complexity-benchmark — 6,000 doubly balanced queries across 10 domains × 3 complexity classes (200/domain/class hard cap; 4,800 train / 1,200 test, 80/20 stratified split, seed=42).
Sources: Derived from sentence-transformers/stackexchange-duplicates (Apache 2.0), cais/mmlu (MIT), TIGER-Lab/MMLU-Pro (MIT), and qiaojin/PubMedQA (MIT).
Hyperparameters:
| Parameter | Value |
|---|---|
| Base model | answerdotai/ModernBERT-base |
| Epochs | 5 |
| Batch size | 32 |
| Learning rate | 2e-5 |
| Max sequence length | 128 tokens |
| Optimizer | AdamW, weight_decay=0.01 |
| Warmup | 10% of steps |
| Best model metric | macro-F1 |
Evaluation
Evaluated on a fixed 750-query held-out test set (250/class), stratified split, seed=42.
| Metric | Value |
|---|---|
| Accuracy | 64.2% |
| Macro-F1 | 0.640 |
| FREE-tier retention | 85.4% |
| Latency p50 (CPU) | 32 ms |
Per-class recall (Wilson 95% CI):
| Class | Recall | 95% CI |
|---|---|---|
| LOW | 70.8% | [66.1%, 75.0%] |
| MEDIUM | 49.3% | [44.4%, 54.1%] |
| HIGH | 72.5% | [67.9%, 76.7%] |
Judge Comparison
| Judge | Latency p50 | Accuracy | Macro-F1 | API dependency |
|---|---|---|---|---|
| ModernBERT (this model) | 32 ms | 64.2% | 0.640 | None |
| Llama 3.2 3B (LLM judge) | 164 ms | 49.0% | 0.436 | Ollama |
Threshold-Tunable Routing
Rather than a fixed argmax decision, STREAM exposes a tunable threshold θ ∈ [0,1]. A query is routed to cloud when P(HIGH) ≥ θ; otherwise to HPC or local. As θ increases, cloud spend drops but HIGH recall decreases — a continuous precision-recall-cost tradeoff.
Budget-aware adaptive routing automatically raises θ as cloud spend approaches the monthly budget cap:
θ_eff(t) = max(θ_base, S(t)/B)
where S(t) is cumulative spend and B is the monthly budget.
Integration in STREAM
from stream.middleware.core.complexity_judge import judge_complexity
result = judge_complexity("Explain quantum entanglement", strategy="modernbert")
# JudgmentResult(complexity='medium', method='classifier', strategy_used='modernbert',
# scores={'low': 0.08, 'medium': 0.79, 'high': 0.13})
Citation
@inproceedings{nassar2026stream,
title = {{STREAM}: Multi-Tier {LLM} Inference Middleware with Dual-Channel {HPC} Token Streaming},
author = {Nassar, Anas and Mohr, Steve and Apanasevich, Leonard and Sharma, Himanshu},
booktitle = {Practice and Experience in Advanced Research Computing (PEARC '26)},
year = {2026},
doi = {10.1145/3785462.3815847}
}
@misc{nassar2026benchmark,
author = {Nassar, Anas},
title = {{LLM} Query Complexity Benchmark},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/datasets/anasnassar/llm-query-complexity-benchmark}
}
% Original source datasets
@article{hendrycks2021mmlu,
author = {Dan Hendrycks and others},
title = {Measuring Massive Multitask Language Understanding},
journal = {ICLR},
year = {2021},
url = {https://huggingface.co/datasets/cais/mmlu}
}
@article{wang2024mmlupro,
author = {Yubo Wang and others},
title = {{MMLU-Pro}: A More Robust and Challenging Multi-Task Language Understanding Benchmark},
journal = {arXiv:2406.01574},
year = {2024},
url = {https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro}
}
@inproceedings{jin2019pubmedqa,
author = {Qiao Jin and others},
title = {{PubMedQA}: A Biomedical Research Question Answering Dataset},
booktitle = {EMNLP},
year = {2019},
url = {https://huggingface.co/datasets/qiaojin/PubMedQA}
}
@misc{stackexchange_dataset,
author = {Reimers, Nils and Gurevych, Iryna},
title = {{StackExchange} Duplicate Questions Dataset},
year = {2019},
url = {https://huggingface.co/datasets/sentence-transformers/stackexchange-duplicates}
}
License
Apache 2.0