BERT AI-Text Detector (HC3 reddit_eli5)
bert-base-uncased fine-tuned to classify a text as human-written (label 0) or
AI-generated (label 1). Trained on the HC3 reddit_eli5 subset (human vs ChatGPT/GPT-3.5
answers, 2023), 47.6K samples, class-weighted loss, max_length=256.
| Evaluation | Macro-F1 |
|---|---|
| Held-out reddit_eli5 test (n=10,200) | 0.9935 |
| finance - cross-domain, never trained on (n=8,436) | 0.9812 |
| medicine - cross-domain, never trained on (n=2,582) | 0.9821 |
Known limitations
- Trained on 2023 GPT-3.5 text: newer generators are out of distribution - generator drift is the predicted failure axis, not topic shift.
- The model leans on a length/style prior: abnormally short AI text can evade it, and long, well-structured human writing is what it most often false-flags. Not suitable for high-stakes accusations.
- English only.
Full project - 5-model comparison, cross-domain study, 100%-coverage error audit, live demo: https://github.com/lilhuang15/ai-generated-text-detector