RoBERTuito fine-tuned on CL2 — Hate speech detection in Chilean Spanish
This is a binary hate-speech detector for Chilean Spanish social-media text.
It is RoBERTuito
(a RoBERTa model pre-trained on 500M Spanish tweets) fine-tuned on CL2, a
balanced Chilean Spanish hate-speech corpus. The model outputs hate or
non-hate.
It is the deployment model recommended in the paper Hate speech detection in Chilean Spanish and its cross-lingual transferability, where CL2 is shown to be the most effective training resource for Chilean Spanish and RoBERTuito the most competitive model on the Chilean target.
Intended use
- Detecting hate speech directed at protected groups (women, immigrants, Indigenous peoples, and the LGBTQ+ community) in Chilean Spanish tweets and short social-media messages.
- Best suited to balanced or pre-filtered inputs (e.g. moderation queues). On a raw, low-prevalence stream, tune the decision threshold and see the paper for the trade-offs.
Usage
The model requires the same text normalisation used at training time
(mentions → @usuario, links → url, lowercase). Skipping it degrades results.
import re, torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
_user = re.compile(r"@\w+"); _url = re.compile(r"https?://\S+|www\.\S+")
def preprocess(t):
return _url.sub("url", _user.sub("@usuario", str(t))).lower().strip()
name = "mmendoza/robertuito-cl2-hate-speech"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()
text = "no los soporto, que se vayan de mi pais"
enc = tok(preprocess(text), return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
prob = torch.softmax(model(**enc).logits, -1)[0]
print(model.config.id2label[int(prob.argmax())], f"p(hate)={prob[1]:.3f}")
Labels: {0: "non-hate", 1: "hate"}.
Training data
CL2 — a Chilean Spanish hate-speech corpus of 4,547 tweets collected from the Chilean X (formerly Twitter) network (Jan 2020–Jun 2022), 45.6 % labelled as hate, annotated by three independent annotators (majority vote) within the full conversation thread. It deliberately covers four protected groups: women, immigrants, Native Americans (Indigenous peoples), and the LGBTQ+ community. Available on Zenodo: https://zenodo.org/records/14619078
Training procedure
Fine-tuned on the full CL2 corpus with:
| Hyperparameter | Value |
|---|---|
| Base model | pysentimiento/robertuito-base-uncased |
| Epochs | 5 |
| Batch size | 32 |
| Learning rate | 5e-5 |
| Max sequence length | 128 |
| Weight decay | 0.01 |
| Warmup ratio | 0.1 |
| Precision | fp16 |
| Seed | 42 |
Evaluation
In-domain (held-out 20 % of CL2, stratified, n = 910):
| Accuracy | Precision | Recall | F1 | AUC |
|---|---|---|---|---|
| 0.864 | 0.833 | 0.877 | 0.854 | 0.939 |
(These figures come from a held-out split; the released weights are trained on the full CL2 corpus for deployment.)
Functional tests (Multilingual HateCheck, Spanish, 27 functionalities): the CL2-trained model reaches the best average accuracy among the compared corpora (0.74), matching the XLM-T reference and surpassing or tying it on 14 of the 27 functionalities. It exhibits low unintended bias across seven demographic groups. See the paper for full transferability, functional, and bias results.
Limitations and biases
- Scope is Chilean Spanish. The model encodes Chilean localisms and slang; it is not intended as a general or pan-Hispanic hate-speech detector, and transfers less well to other varieties and to English.
- Trained on keyword/hashtag/account-sampled data, which over-represents explicit hate; implicit or subtly framed hate is harder to detect.
- The corpus reflects the annotators' local judgement and the four targeted groups; other targets are out of scope.
- Predictions can err on profanity-heavy but non-hateful colloquial speech and on non-hateful references to protected groups. Use with human review for consequential decisions.
Citation
@article{benoit_hate_chilean_spanish,
title = {Hate speech detection in Chilean Spanish and its cross-lingual transferability},
author = {Benoit, Domingo and {\~N}anculef, Ricardo and Mendoza, Marcelo},
journal = {International Journal of Data Science and Analytics (under review)},
year = {2026}
}
Please also cite the base model (Pérez et al., RoBERTuito, LREC 2022) and the CL2 corpus (Zenodo 14619078).