ReviewMate Rhetorical Sentence Classifier — v1
A DistilBERT model that classifies sentences from research paper abstracts into five rhetorical roles. Optimized for empirical scientific writing across STEM domains, with strong performance on computer science / AI literature.
Task
Given a sentence from a research abstract, predict its rhetorical role:
BACKGROUND— context, prior work, motivationOBJECTIVE— research goal or hypothesisMETHODS— approach, design, techniques usedRESULTS— findings, benchmarks, measurementsCONCLUSIONS— interpretation, implications, future work
These rhetorical categories apply to empirical scientific writing across most STEM domains.
Performance
Evaluated on held-out test sets:
| Test Set | Domain | Accuracy | F1 Macro | F1 Weighted |
|---|---|---|---|---|
| CSAbstruct | CS / AI | 0.7517 | 0.7556 | 0.7527 |
| PubMed-RCT | Biomedical | 0.8833 | 0.8294 | 0.8825 |
Best suited for computer science, AI/ML, applied STEM, and quantitative empirical research. Performance may degrade on theoretical, mathematical, or humanities papers due to different rhetorical conventions.
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "Himel000/reviewmate-classifier-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
sentence = "We propose a novel transformer architecture for sequence classification."
inputs = tokenizer(sentence, return_tensors="pt", truncation=True, max_length=64)
with torch.no_grad():
outputs = model(**inputs)
predicted_class_id = outputs.logits.argmax(dim=-1).item()
predicted_label = model.config.id2label[predicted_class_id]
print(predicted_label)
Training Approach
The model was fine-tuned in two sequential stages to leverage large-scale rhetorical data while adapting to target domain writing style.
Stage 1 — Large-scale rhetorical learning:
- Base:
distilbert-base-uncased - Data: PubMed-RCT-200k (~2.2M labeled sentences)
- Epochs: 2, learning rate: 5e-5, effective batch size: 128
- Purpose: Learn general rhetorical patterns from the largest available labeled dataset
Stage 2 — Domain adaptation:
- Continued fine-tuning from Stage 1 weights
- Data: CSAbstruct (~11k labeled CS sentences)
- Epochs: 4, learning rate: 2e-5 (low to preserve Stage 1 knowledge), batch size: 32
- Purpose: Adapt rhetorical patterns to CS/AI writing conventions
Total compute: ~4 hours (Kaggle T4 x2).
The two-stage approach improved out-of-domain F1 from 0.39 (after Stage 1 alone) to 0.76, demonstrating the value of staged transfer learning for cross-domain rhetorical classification.
Project Context
This model powers the rhetorical extraction component of ReviewMate, an AI-powered literature scanning and synthesis tool for empirical research papers.
Scope and Limitations
- Optimized for empirical scientific writing (introduction → method → result → conclusion structure)
- Strongest on CS, AI/ML, applied STEM, and quantitative empirical research
- Not designed for theoretical mathematics (theorem-proof structure), humanities, or non-empirical writing
- Single-sentence classification (no surrounding sentence context)
- Cross-domain expansion to additional fields is part of the project's open contribution roadmap
Citation
Datasets used for training:
Dernoncourt, F., & Lee, J. Y. (2017). PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts. IJCNLP 2017.
Cohan, A., Beltagy, I., King, D., Dalvi, B., & Weld, D. (2019). Pretrained Language Models for Sequential Sentence Classification. EMNLP 2019.