GPT-Neo-1.3B Toxicity Surrogate Scorer
A GPTNeoForSequenceClassification head fine-tuned on the Jigsaw Toxic Comment
Classification corpus. It outputs a continuous P(toxic) ∈ [0, 1] for a piece
of English text (softmax probability of the toxic class, label index 1).
The model is used as an external violation scorer (a CDD-style surrogate) in research on guided discrete diffusion for controllable text generation: at evaluation time it scores decoded samples, and it also provides the soft labels used to train the per-timestep guidance classifiers.
Intended use
- Primary: reward / violation scoring for controllable generation experiments.
- Input: a raw English string. Output:
softmax(logits)[1]= P(toxic). - Not intended as a production content-moderation system.
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "TheSky0908/gpt-neo-1.3B-toxicity-surrogate"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
text = "I hope you have a wonderful day."
enc = tok(text, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
p_toxic = torch.softmax(model(**enc).logits, dim=-1)[0, 1].item()
print(f"P(toxic) = {p_toxic:.4f}")
Training data
Arsive/toxicity_classification_jigsaw.
The six Jigsaw sub-labels (toxic, severe_toxic, obscene, threat,
insult, identity_hate) are consolidated into a single binary target:
label 1 if any sub-label is positive, else 0.
Training procedure
- Base model:
EleutherAI/gpt-neo-1.3B, 2-way sequence-classification head. - Objective: cross-entropy on the binarized toxic label.
- Optimizer: AdamW, lr
1e-5, weight decay0.01, linear schedule with 6% warmup, grad-clip 1.0. - Batch size: 16 · Max length: 256 tokens · Epochs: 3 (~4.9k steps).
- Precision: bf16 autocast, fp32 weights · gradient checkpointing enabled.
- Best checkpoint selected by validation F1 (early stopping on the full val set).
Evaluation (Jigsaw validation split)
| Metric | Value |
|---|---|
| F1 | 0.9885 |
| Accuracy | 0.9886 |
| ROC-AUC | 0.9986 |
| Precision | 0.9855 |
| Recall | 0.9916 |
Limitations & bias
Trained only on Jigsaw English comments, so it inherits that corpus's annotation biases (e.g. sensitivity to identity terms) and does not transfer to other languages or domains. High held-out F1 reflects in-distribution Jigsaw data, not real-world moderation performance. Use as a research surrogate signal, not as a safety-critical filter.