modernbert-base-emotions

Overview

This model was fine-tuned from ModernBERT-base on the Super Emotion dataset for multi-class emotion classification. It predicts emotional states in text across seven labels: joy, sadness, anger, fear, love, neutral, surprise.

Model Details

• Base Model: ModernBERT-base
• Fine-Tuning Dataset: Super Emotion
• Number of Labels: 7
• Problem Type: Single-label classification
• Language: English
• License: CC-BY-4.0
• Fine-Tuning Framework: Hugging Face Transformers

Example Usage

Here’s how to use the model with Hugging Face Transformers:

from transformers import pipeline

# Load the model
classifier = pipeline(
    "text-classification", 
    model="cirimus/modernbert-base-emotions",
    top_k=5
)

text = "I can't believe this just happened!"
predictions = classifier(text)

# Print top 3 detected emotions
sorted_preds = sorted(predictions[0], key=lambda x: x['score'], reverse=True)
top_3 = sorted_preds[:3]

print("\nTop 3 emotions detected:")
for pred in top_3:
    print(f"\t{pred['label']:10s} : {pred['score']:.3f}")

# Example output:
# Top 3 emotions detected:
#        SURPRISE   : 0.913
#        SADNESS    : 0.033
#        NEUTRAL    : 0.021

How the Model Was Created

The model was fine-tuned for 2 epochs using the following hyperparameters:

• Learning Rate: 2e-5
• Batch Size: 16
• Weight Decay: 0.01
• Warmup Steps: Cosine decay scheduling
• Optimizer: AdamW
• Evaluation Metrics: Precision, Recall, F1 Score (macro), Accuracy

Evaluation Results

As evaluated on the joint test-set:

Intended Use

The model is designed for emotion classification in English-language text, particularly useful for:

• Social media sentiment analysis
• Customer feedback evaluation
• Large scale behavioral or psychological research

The model is designed for fast and accurate emotion detection but struggles with subtle expressions or indirect references to emotions (e.g., "I find myself remembering the little things you say, long after you've said them.")

Limitations and Biases

• Data Bias: The dataset is aggregated from multiple sources and may contain biases in annotation and class distribution.
• Underrepresented Classes: Some emotions have fewer samples, affecting their classification performance.
• Context Dependence: The model classifies individual sentences and may not perform well on multi-sentence contexts.

Environmental Impact

• Hardware Used: NVIDIA RTX 4090
• Training Time: < 1 hour
• Carbon Emissions: ~0.04 kg CO2 (estimated via ML CO2 Impact Calculator)

Citation

If you use this model, please cite:

@inproceedings{JdFE2025b,
  title = {Emotion Detection with ModernBERT},
  author = {Enric Junqu\'e de Fortuny},
  year = {2025},
  howpublished = {\url{https://huggingface.co/your_model_name_here}},
}