turkish-emotion-classifier
A fine-tuned version of dbmdz/bert-base-turkish-cased
for multi-class emotion classification of Turkish text.
Model description
The model takes a short Turkish text and predicts one of six emotion labels. It
is a standard BertForSequenceClassification head fine-tuned on top of the cased
Turkish BERT (BERTurk) encoder.
- Base model:
dbmdz/bert-base-turkish-cased - Language: Turkish (
tr) - Task: Single-label, multi-class text classification (emotion)
- Number of classes: 6
Labels
The label ids follow model.config.id2label:
| id | label |
|---|---|
| 0 | anger |
| 1 | disgust |
| 2 | fear |
| 3 | joy |
| 4 | sadness |
| 5 | surprise |
Intended uses & limitations
Intended use: classifying the dominant emotion of short, informal Turkish sentences (reviews, social posts, messages).
The model expects raw text — no special preprocessing is required. Just pass your sentence directly; the tokenizer (shipped with the model) handles everything.
Limitations / out of scope:
- Long, multi-sentence documents. The model was trained on short single sentences and predicts a single dominant emotion. For longer text, split into sentences and classify each one separately.
- Formal/legal text, languages other than Turkish, and fine-grained sentiment beyond the six trained labels.
How to use
from transformers import pipeline
pipe = pipeline("text-classification", model="furkankarakuz/turkish-emotion-classifier")
print(pipe("Bu film gerçekten harikaydı!"))
# [{'label': 'joy', 'score': 0.99}]
Or load the model and tokenizer directly:
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("furkankarakuz/turkish-emotion-classifier")
tokenizer = AutoTokenizer.from_pretrained("furkankarakuz/turkish-emotion-classifier")
Training and evaluation data
Trained on a manually labeled Turkish emotion dataset of roughly 23,000 samples covering six emotions (anger, disgust, fear, joy, sadness, surprise). The data was split with stratification so the class balance is preserved across splits:
- Train: 70%
- Validation: 15%
- Test: 15%
Labels were integer-encoded with LabelEncoder, fit on the training split only.
No lossy text cleaning was applied; the model is trained on raw text.
Training procedure
Fine-tuned with the 🤗 Transformers Trainer, using a class-weighted
cross-entropy loss to handle class imbalance. The best checkpoint was selected on
the validation macro-F1 (best at epoch 8) and restored as the final model.
Training hyperparameters
| Hyperparameter | Value |
|---|---|
| Base model | dbmdz/bert-base-turkish-cased |
| Max sequence length | 64 (dynamic padding) |
| Train batch size | 32 |
| Eval batch size | 64 |
| Learning rate | 2e-5 |
| LR scheduler | linear, warmup ratio 0.1 |
| Optimizer | AdamW (torch fused), betas=(0.9, 0.999), eps=1e-8 |
| Epochs | 10 (early stopping, patience 2) |
| Loss | class-weighted cross-entropy |
| Best-model metric | f1_macro (validation) |
| Mixed precision | native AMP |
| Seed | 42 |
Validation results per epoch
The best checkpoint (epoch 8, by validation macro-F1) was restored as the final model.
| Epoch | Training Loss | Validation Loss | Accuracy | F1 (macro) | Precision (macro) | Recall (macro) |
|---|---|---|---|---|---|---|
| 1 | 0.1866 | 0.1858 | 0.9460 | 0.9460 | 0.9481 | 0.9442 |
| 2 | 0.1412 | 0.1436 | 0.9591 | 0.9583 | 0.9575 | 0.9592 |
| 3 | 0.0766 | 0.1639 | 0.9619 | 0.9617 | 0.9621 | 0.9619 |
| 4 | 0.0364 | 0.1749 | 0.9622 | 0.9616 | 0.9627 | 0.9608 |
| 5 | 0.0269 | 0.1892 | 0.9631 | 0.9620 | 0.9609 | 0.9635 |
| 6 | 0.0132 | 0.1901 | 0.9656 | 0.9650 | 0.9652 | 0.9649 |
| 7 | 0.0141 | 0.1980 | 0.9642 | 0.9635 | 0.9648 | 0.9625 |
| 8 | 0.0011 | 0.1990 | 0.9676 | 0.9673 | 0.9670 | 0.9676 |
| 9 | 0.0063 | 0.2036 | 0.9665 | 0.9657 | 0.9661 | 0.9656 |
| 10 | 0.0043 | 0.2022 | 0.9670 | 0.9664 | 0.9667 | 0.9662 |
Evaluation results
Results on the held-out test set (3,520 samples), using the final model:
| Metric | Value |
|---|---|
| Accuracy | 0.9690 |
| F1 (macro) | 0.9683 |
| Precision (macro) | 0.9676 |
| Recall (macro) | 0.9691 |
| Loss | 0.1880 |
Per-class results (test set)
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| anger | 0.96 | 0.96 | 0.96 | 616 |
| disgust | 0.96 | 0.98 | 0.97 | 419 |
| fear | 0.98 | 0.97 | 0.97 | 623 |
| joy | 0.98 | 0.98 | 0.98 | 783 |
| sadness | 0.96 | 0.97 | 0.96 | 662 |
| surprise | 0.96 | 0.96 | 0.96 | 417 |
Limitations and bias
The model reflects the distribution and any biases of its training data and may underperform on dialects, domains, or emotion expressions not well represented in it. Predictions are most reliable for short, informal text similar to the training data.
Framework versions
- Transformers 5.9.0
- PyTorch 2.11.0+cu128
- Datasets 5.0.0
- Tokenizers 0.22.2