DeBERTa-v3-Large for Emotion Detection (Merged & Augmented Dataset)

This model is fine-tuned on microsoft/deberta-v3-large on a merged and augmented version of the following datasets:

• 🤗 GoEmotions
• 📘 ISEAR Dataset
• 📙 Emotion Dataset (DAIR-AI)

The model is trained for 7-class emotion classification in English and achieves state-of-the-art performance using advanced augmentation and weighted loss.

🧠 Emotion Classes

• 😠 anger
• 🤢 disgust
• 😨 fear
• 😀 happy
• 😐 neutral
• 😢 sad
• 😲 surprise

📈 Training Metrics

🛠️ Training Configuration

training_args = TrainingArguments(
    output_dir="./deberta-large-3-merged_augmented",
    eval_strategy="epoch",
    save_strategy="epoch",
    learning_rate=1e-5,
    per_device_train_batch_size=32,
    per_device_eval_batch_size=32,
    gradient_accumulation_steps=2,
    num_train_epochs=20,
    weight_decay=0.01,
    lr_scheduler_type="cosine",
    logging_dir="./logs",
    logging_steps=50,
    save_total_limit=1,
    load_best_model_at_end=True,
    metric_for_best_model="accuracy",
    report_to="none",
    dataloader_num_workers=8
)

🔄 Confusion Matrix

📊 Classification Report

🔧 How to Use

from transformers import DebertaV2Tokenizer, DebertaV2ForSequenceClassification
import torch

text = "I'm feeling very nervous about tomorrow."

tokenizer = DebertaV2Tokenizer.from_pretrained('Tanneru/Emotion-Classification-DeBERTa-v3-Large')
model = DebertaV2ForSequenceClassification.from_pretrained('Tanneru/Emotion-Classification-DeBERTa-v3-Large')


inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predicted_class_id = torch.argmax(outputs.logits).item()

print("Predicted emotion:", model.config.id2label[predicted_class_id])

📄 License

This model is released under the Apache 2.0 License. You are free to use, modify, and distribute the model with proper attribution.

✍️ Author

• Username: Tanneru
• Base model: microsoft/deberta-v3-large

📚 Citation

If you use this model in your work, please cite:

@misc{tanneru2025deberta_emotion,
  title={DeBERTa-v3-Large fine-tuned on Merged & Augmented Emotion Datasets},
  author={Tanneru},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Tanneru/Emotion-Classification-DeBERTa-v3-Large}},
}

@article{he2021deberta,
  title={DeBERTa: Decoding-enhanced BERT with Disentangled Attention},
  author={He, Pengcheng and Liu, Xiaodong and Gao, Jianfeng and Chen, Weizhu},
  journal={arXiv preprint arXiv:2006.03654},
  year={2021}
}