A lightweight binary toxicity classifier for Russian-language text, fine-tuned from cointegrated/rubert-tiny on the AlexSham/Toxic_Russian_Comments dataset.
The model outputs a single binary label — toxic (1) or non-toxic (0) — and is designed to be fast and small enough for real-time moderation pipelines (chat filters, comment sections, forums).
Training Approach
- The full dataset was used as-is — no downsampling of the majority class.
- Class imbalance was handled via class weights in
CrossEntropyLoss(computed withsklearn.utils.class_weight.compute_class_weight(class_weight="balanced")), which upweights the toxic class roughly ~4.5x relative to the non-toxic class. - Trained with Hugging Face
Trainer(4 epochs, batch size 64, lr 3e-5, warmup ratio 0.06), selecting the best checkpoint by F1 on the toxic class (not accuracy, which is misleading under this imbalance).
Evaluation
Evaluated on the held-out test split of AlexSham/Toxic_Russian_Comments:
| Model | Toxic F1 |
|---|---|
| BazarFiltr-0.1 | 0.907 |
| cointegrated/rubert-tiny-toxicity | 0.73 |
Note: cointegrated/rubert-tiny-toxicity was trained on a different dataset and under a different problem formulation (multilabel), so this comparison is indicative rather than a strictly controlled benchmark — but it reflects real-world performance on this specific data distribution.
Known Limitations
- The model catches explicit profanity, insults, and threats.
- It can miss disguised or "politely phrased" toxicity — including racist statements that don't use profanity or obvious slurs. Use with caution in contexts where subtle hate speech is a concern, and consider pairing with human review or additional filters for sensitive applications.
Usage
Inputs are truncated to a maximum of 128 tokens.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "akanemind/BazarFiltr-0.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
text = "Ты вообще нормальный человек или нет?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
logits = model(**inputs).logits
probs = torch.softmax(logits, dim=-1)
pred = torch.argmax(probs, dim=-1).item()
label = "toxic" if pred == 1 else "non-toxic"
print(f"{label} (p_toxic={probs[0][1]:.3f})")
Intended Use
- Filtering/flagging toxic comments in Russian-language chats, forums, and social platforms
- Pre-moderation triage (flag-for-review, not fully automated takedown, given the limitations above)
- Research on Russian toxicity detection
Out of Scope
- Non-Russian text
- Fine-grained toxicity categories (this is binary, not multilabel like insult/threat/obscenity breakdowns)
- Standalone moderation without human oversight for high-stakes decisions
Training Data
AlexSham/Toxic_Russian_Comments — Russian comments labeled toxic/non-toxic, with a natural class imbalance of roughly 82% non-toxic to 18% toxic.
| Metric | BazarFiltr-0.1 | rubert-tiny-toxicity |
|---|---|---|
| Accuracy | 0.965 | 0.866 |
| Toxic precision | 0.86 | 0.57 |
| Toxic recall | 0.96 | 0.99 |
| Toxic F1 | 0.907 | 0.73 |
| Non-toxic F1 | 0.978 | 0.91 |
The baseline's high recall comes at the cost of precision — it flags far more non-toxic text as toxic (699 vs. 3285 false positives), which drags its F1 down despite catching nearly all toxic examples.