Model Card for ethicalabs/Echo-DSRN-v0.1.4-Research-Intent-CLF
6-class research paper intent classifier built on the Echo-DSRN architecture (98.3M params), fine-tuned from ethicalabs/Echo-DSRN-v0.1.3-Research-Intent-CLF (5-class).
Labels: Methodology · Dataset · Review · Applied · Theoretical · Unclassifiable
The Unclassifiable class is new in v0.1.4 — papers flagged by >1 LLM judge as garbled / non-English / non-academic.
OpenAIRE AI Hackathon 2026 🇪🇺
This model is part of ethicalabs.ai's entry in the OpenAIRE AI Hackathon 2026, co-organised by OpenAIRE and Alien Intelligence — a 12-week open science build challenge.
🔴 Live Demo → openaire-2026.ethicalabs.ai
The live application classifies research papers in real time using this model, with publication metadata streamed from the OpenAIRE Graph API under CC BY 4.0.
Training
- Data:
ethicalabs/Research-Intent-Collab(10,001 papers, LLM-judge majority consensus + human curation). Officialtrain(7,999) /validation(1,001) /test(1,001) splits. - Recipe: head swap 5→6, differential AdamW (backbone
2e-5, head2e-4), batch 16, 3 epochs (1,500 steps), bf16, cosine LR + 5% warmup, early stopping on the official validation split.
Results (official test split, 1,001 rows)
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Methodology | 0.8140 | 0.7883 | 0.8009 | 222 |
| Dataset | 0.7097 | 1.0000 | 0.8302 | 22 |
| Review | 0.6400 | 0.5845 | 0.6110 | 219 |
| Applied | 0.7162 | 0.8112 | 0.7608 | 392 |
| Theoretical | 0.7333 | 0.7500 | 0.7416 | 44 |
| Unclassifiable | 0.9091 | 0.5882 | 0.7143 | 102 |
| Macro avg | 0.7537 | 0.7537 | 0.7431 | 1001 |
Accuracy: 0.7353 (736/1001).
Comparison with v0.1.3 (same official test split)
v0.1.3 (5-class) evaluates on 899 rows only — its head cannot produce Unclassifiable, so all 102 such test rows would be misclassified.
v0.1.4 covers all 1,001 rows and catches 60 of the 102 with 91% precision.
| Metric | v0.1.3 (5-class) | v0.1.4 (6-class) |
|---|---|---|
| Accuracy, all evaluated rows | 0.7197 (647/899) | 0.7353 (736/1001) |
| Accuracy, 5-class subset | 0.7197 (647/899) | 0.7519 (676/899) |
| Macro F1, 5-class subset | 0.7328 | 0.7489 |
| Macro F1, all 6 classes | — | 0.7431 |
| Unclassifiable | — (all 102 misclassified) | F1 0.7143 (P 0.909) |
Per-class F1 (test split):
| Label | v0.1.3 | v0.1.4 | Δ |
|---|---|---|---|
| Methodology | 0.7563 | 0.8009 | +0.045 |
| Dataset | 0.8627 | 0.8302 | −0.033 |
| Review | 0.5876 | 0.6110 | +0.023 |
| Applied | 0.7524 | 0.7608 | +0.008 |
| Theoretical | 0.7048 | 0.7416 | +0.037 |
| Unclassifiable | — | 0.7143 | new |
v0.1.4 gains accuracy on the original classes (+3.2 pts on the 5-class subset) and adds the Unclassifiable safety class.
Review remains the weakest class for both models (mostly confused with Applied).
Usage
from echo_dsrn import pipeline
# Requires echo_dsrn >= 0.1.6 (chat-template support for
# text-classification pipelines). Tiny model — CPU load + inference
# is faster than GPU for this size.
pipe = pipeline(
"text-classification",
model="ethicalabs/Echo-DSRN-v0.1.4-Research-Intent-CLF",
trust_remote_code=True,
device="cpu",
)
# Raw title+abstract string — the pipeline applies the model's chat
# template (system prompt + user phrasing) automatically.
pipe(
"Title: Attention Is All You Need\n"
"Abstract: The dominant sequence transduction models are based on complex "
"recurrent or convolutional neural networks that include an encoder and a "
"decoder. The best performing models also connect the encoder and decoder "
"through an attention mechanism. We propose a new simple network "
"architecture, the Transformer, based solely on attention mechanisms, "
"dispensing with recurrence and convolutions entirely. Experiments on two "
"machine translation tasks show these models to be superior in quality "
"while being more parallelizable and requiring significantly less time to "
"train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German "
"translation task, improving over the existing best results, including "
"ensembles, by over 2 BLEU."
)
# [{'label': 'Methodology', 'score': 0.9518}]
Made in 🇪🇺 with 💗 for Open Science 🤗 - github.com/ethicalabs-ai/OpenAIRE-AI-Research-Evaluator