Polemo Intensity Model
This model is fine-tuned for emotion intensity detection in Polish political texts. It is based on the RoBERTa transformer model (https://github.com/sdadas/polish-roberta) and has been specifically trained to recognize a range of emotions in text data scraped from various social media platforms. The model returns 0 to 1 continuous scores for eight emotional metrics: Happiness, Sadness, Anger, Disgust, Fear, Pride, Valence, and Arousal.
Disclaimer
This model was updated on 7/8/2025. The previous version was corrupted and gave mostly random estimates of emotion intensity. If you used the model before that, sorry for the inconvenience.
Model Details
Database Preparation
Our research utilizes a comprehensive database of Polish political texts from social media profiles (i.e., YouTube, Twitter, Facebook) of 25 journalists, 25 politicians, and 19 non-governmental organizations (NGOs). For each profile, all available posts from each platform were scraped going back to the beginning of 2019. In addition, we included texts written by non-professional commentators of social affairs. Our dataset consists of 1,246,337 text snippets:
- Twitter: 789,490 tweets
- YouTube: 42,252 comments
- Facebook: 414,595 posts
The texts were processed to fit transformer models' length constraints. Facebook texts were split into sentences, and all texts longer than 280 characters were removed. Non-Polish texts were filtered out using the langdetect software, and all online links and usernames were replaced with placeholders. We focused on texts with higher emotional content for training, which we have filtered using a lexicon approach, resulting in a final dataset of 10,000 texts, annotated by 20 expert annotators.
Annotation Process
The final dataset was annotated for the following emotions:
- Happiness
- Sadness
- Anger
- Disgust
- Fear
- Pride
- Valence
- Arousal
Annotators used a 5-point scale for each emotion and dimension. The annotation process ensured consistency and minimized subjectivity, with each text being annotated by five different annotators.
Model Training
We considered two base models: the Trelbert transformer model and the Polish Roberta model. The final model relied on the Roberta transformer model and was fine-tuned using a Bayesian grid search for hyperparameter optimization. The training involved:
- Dropout: 0.6
- Learning rate: 5e-5
- Weight decay: 0.3
- Warmup steps: 600
Results
The model demonstrated strong correlations with human ratings, particularly in predicting happiness and valence, achieving correlations of 0.87. The results for other emotions were also substantial, indicating the model's ability to capture a wide range of emotional states.
K-Fold Validation
A 10-fold cross-validation showed high reliability across different emotional dimensions:
- Happiness: 0.83
- Sadness: 0.68
- Anger: 0.81
- Disgust: 0.75
- Fear: 0.67
- Pride: 0.76
- Valence: 0.84
- Arousal: 0.71
Usage
You can use the model and tokenizer as follows:
First run the bash code below to clone the repository (this will take some time). Because of the custom model class, this model cannot be run with the usual huggingface Model setups.
git lfs clone https://huggingface.co/hplisiecki/polemo_intensity
Proceed as follows:
from polemo_intensity.model_script import Model # importing the custom model class
from transformers import AutoTokenizer
model_directory = "C:placeholder/polemo_intensity" # Your full path to the model's directory
model = Model.from_pretrained(model_directory)
tokenizer = AutoTokenizer.from_pretrained(model_directory)
inputs = tokenizer("This is a test input.", return_tensors="pt")
outputs = model(inputs['input_ids'], inputs['attention_mask'])
# Print out the emotion ratings
for emotion, rating in zip(['Happiness', 'Sadness', 'Anger', 'Disgust', 'Fear', 'Pride', 'Valence', 'Arousal'], outputs):
print(f"{emotion}: {rating.item()}")
Team
The model was created at the Digital Social Sciences Research Laboratory at IFIS PAN https://lab.ifispan.edu.pl
Citation
Please cite the following paper when using this model: https://doi.org/10.1007/s11135-025-02116-8