PaperGuard AI Detector (v2.0)
Fine-tuned DistilBERT (distilbert-base-cased) sequence classifier that
detects AI-generated text in academic papers, essays, and reports. It is the
core AI-detection engine of the PaperGuard
multi-agent academic-integrity system.
Labels
0 = ai, 1 = human (see config.json id2label).
Training data
v2.0 ("mega") continues from the v1.5 checkpoint and adds a larger, more diverse mix so the model sees many modern LLM writing styles:
- Claude Opus distillation — recent Anthropic-style generations
- Ateeqq/AI-and-Human-Generated-Text — academic abstracts (human + AI)
- artem9k/ai-text-detection-pile — large open human-vs-AI corpus
- (v1.5 lineage) Defactify + Ateeqq, spanning 30+ frontier LLMs (GPT-4o, LLaMA-3, Claude 3, Gemini, Mistral, Qwen, ChatGPT, …)
Important: use the logit margin, not the raw softmax
On easy/separable data the model becomes overconfident — its softmax
saturates (it can report ~0% AI even on genuine AI text). The discriminative
signal lives in the logit margin (human_logit − ai_logit). PaperGuard
therefore scores AI-likelihood from a logistic calibration of the margin,
not the raw softmax. After calibration it flags clean/academic AI at ~70–90%
while keeping human text low (~10%).
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tok = AutoTokenizer.from_pretrained("vediumsameer/paperguard-ai-detector")
model = AutoModelForSequenceClassification.from_pretrained("vediumsameer/paperguard-ai-detector")
text = "The rapid advancement of artificial intelligence has transformed modern education..."
inputs = tok(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
logits = model(**inputs).logits[0]
# Recommended: score off the margin (softmax is saturated)
margin = float(logits[model.config.label2id["human"]] - logits[model.config.label2id["ai"]])
# lower margin -> more AI-like ; higher margin -> more human-like
print("logit margin (human - ai):", margin)
Limitations
- Blind spot: slang / style-masked AI (an LLM told to write casually) can still read as human. PaperGuard mitigates this with embedding-based stylometric "patchwork" detection, and a v2.1 trained on adversarial / multi-model / reasoning data is planned to close the gap.
- AI detection is a probabilistic indicator, not proof of authorship.
Part of PaperGuard
This model powers the AI-detection layer of PaperGuard, which also does citation claim verification, plagiarism, and writing-quality analysis. See the project repo.