ME²-BERT: Are Events and Emotions what you need for Moral Foundation Prediction?
Moralities, emotions, and events are complex aspects of human cognition, which are often treated separately since capturing their combined effects is challenging, especially due to the lack of annotated data. Leveraging their interrelations hence becomes crucial for advancing the understanding of human moral behaviors. In this work, we propose ME²-BERT, the first holistic framework for fine-tuning a pre-trained language model like BERT to the task of moral foundation prediction. ME²-BERT integrates events and emotions for learning domain-invariant morality-relevant text representations. Our extensive experiments show that ME²-BERT outperforms existing state-of-the-art methods for moral foundation prediction, with an average percentage increase up to 35% in the out-of-domain scenario.
Paper | Source code | WebApp
Training Data
ME²-BERT was fine-tuned on the E2MoCase dataset (available upon request), which consists of 97,251 paragraphs from news articles encompassing both event-based and event-free samples. It includes annotations for:
- Moral values: Care, Harm, Fairness, Cheating, Loyalty, Betrayal, Authority, Subversion, Purity, Degradation.
- Emotions: Anticipation, Trust, Disgust, Joy, Optimism, Surprise, Love, Anger, Sadness, Pessimism, Fear.
- Events in JSON format, including the trigger mention and the entities involved in the event.
Evaluation data
ME²-BERT has been evaluated on:
- Moral Foundation Twitter Corpus (MFTC)
- Moral Foundation Reddit Corpus (MFRC)
- Extended Moral Foundation Dictionary (eMFD)
- MoralEvents
Usage
NOTE: The following examples were executed using Transformers v 4.47.1. If you encounter any issues running our code, please contact us..
from transformers import AutoTokenizer, AutoModel
import torch
model_name = "lorenzozan/ME2-BERT"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
text = ["Faithless is he that says farewell when the road darkens."]
inputs = tokenizer(text, padding="max_length", truncation=True, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs, return_dict=False)
print(outputs) # tensor([[0.0185, 0.2401, 0.9166, 0.0498, 0.0453]])
By running the model with return_dict=True, it returns a dictionary containing key-value pairs, where each key represents a moral dimension and its corresponding value indicates the associated score.
text = [
'Faithless is he that says farewell when the road darkens.',
'The soul is healed by being with children.',
'I remembered how we had we had all come to Gatsby’s and guessed at his corruption… while he stood before us concealing an incorruptible dream…',
'All the variety, all the charm, all the beauty of life is made up of light and shadow, but justice must always remain clear and unbroken.',
'When tyranny becomes law, rebellion becomes duty.']
max_seq_length = 200
mf_mapping = {'CH':'CARE/HARM','FC':'FAIRNESS/CHEATING', 'LB':'LOYALTY/BETRAYAL', 'AS':'AUTHORITY/SUBVERSION', 'PD': 'PURITY/DEGRADATION'}
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
encoded_input = tokenizer(
text,
max_length=max_seq_length,
padding="max_length",
truncation=True,
return_tensors="pt",
)
input_ids = encoded_input["input_ids"]
attention_mask = encoded_input["attention_mask"]
model.eval()
with torch.no_grad():
output = model(input_ids=input_ids, attention_mask=attention_mask, return_dict=True)
for i, tt in enumerate(text):
print(tt)
for mf, score in output[i].items():
print(f'{mf_mapping[mf]} : {score}')
print()
Faithless is he that says farewell when the road darkens.
CARE/HARM : 0.05056
FAIRNESS/CHEATING : 0.01845
LOYALTY/BETRAYAL : 0.8676
AUTHORITY/SUBVERSION : 0.01655
PURITY/DEGRADATION : 0.06524
The soul is healed by being with children.
CARE/HARM : 0.83783
FAIRNESS/CHEATING : 0.02016
LOYALTY/BETRAYAL : 0.42663
AUTHORITY/SUBVERSION : 0.00525
PURITY/DEGRADATION : 0.61056
I remembered how we had we had all come to Gatsby’s and guessed at his corruption… while he stood before us concealing an incorruptible dream…
CARE/HARM : 0.00676
FAIRNESS/CHEATING : 0.04518
LOYALTY/BETRAYAL : 0.02287
AUTHORITY/SUBVERSION : 0.00545
PURITY/DEGRADATION : 0.64035
All the variety, all the charm, all the beauty of life is made up of light and shadow, but justice must always remain clear and unbroken.
CARE/HARM : 0.08769
FAIRNESS/CHEATING : 0.95034
LOYALTY/BETRAYAL : 0.05768
AUTHORITY/SUBVERSION : 0.00725
PURITY/DEGRADATION : 0.06396
When tyranny becomes law, rebellion becomes duty.
CARE/HARM : 0.1599
FAIRNESS/CHEATING : 0.91123
LOYALTY/BETRAYAL : 0.4824
AUTHORITY/SUBVERSION : 0.96638
PURITY/DEGRADATION : 0.02086
Other examples of usage with different configuration are shown here.
References
If you use this model, please cite:
@inproceedings{zangari-etal-2025-me2,
title = "{ME}2-{BERT}: Are Events and Emotions what you need for Moral Foundation Prediction?",
author = "Zangari, Lorenzo and
Greco, Candida M. and
Picca, Davide and
Tagarelli, Andrea",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-main.638/",
pages = "9516--9532",
abstract = "Moralities, emotions, and events are complex aspects of human cognition, which are often treated separately since capturing their combined effects is challenging, especially due to the lack of annotated data. Leveraging their interrelations hence becomes crucial for advancing the understanding of human moral behaviors. In this work, we propose ME2-BERT, the first holistic framework for fine-tuning a pre-trained language model like BERT to the task of moral foundation prediction. ME2-BERT integrates events and emotions for learning domain-invariant morality-relevant text representations. Our extensive experiments show that ME2-BERT outperforms existing state-of-the-art methods for moral foundation prediction, with an average increase up to 35{\%} in the out-of-domain scenario."
}