EmotiSpace-128
EmotiSpace-128 is a BERT-based hybrid emotion model that maps text into a 128-dimensional emotional embedding space and also predicts GoEmotions-style emotion labels.
It is not only a classifier. The main goal is to produce a reusable emotional latent space where similar emotional meanings are close together, while still exposing readable emotion probabilities.
What it does
Input:
I feel devastated.
Output:
- a normalized 128D emotion embedding
- raw logits
- sigmoid probabilities for emotion labels
Example labels:
sadness
remorse
disappointment
grief
gratitude
Architecture
bert-base-uncased
-> pooled BERT output
-> projection head: 768 -> 256 -> 128
-> normalized 128D EmotiSpace embedding
-> classifier head
-> emotion probabilities
The model uses a custom Transformers architecture:
model_type: bert_emotispace
Load it with trust_remote_code=True.
Training
The model was trained in two passes.
Pass 1 focused on emotion classification using GoEmotions labels.
Pass 2 continued training with a combined objective:
loss = classification_loss + 0.05 * embedding_geometry_loss
The second pass was used to make the 128D embedding space more useful for cosine similarity and custom emotion anchors.
Training used the last 4 BERT layers unfrozen, with early stopping.
Intended use
EmotiSpace-128 is useful for:
- emotion classification
- emotion embeddings
- emotional similarity search
- custom emotion anchors beyond fixed labels
- character mood systems
- dialogue tone control
- TTS/prosody control pipelines
- conversation mood tracking
It can be used in two modes:
Easy mode
Use the emotion probabilities as a normal classifier.
Advanced mode
Use the 128D embedding directly and compare it against custom emotion anchors.
For example, you can define anchors such as:
nostalgia
warmth
calm
playfulness
emotional flatness
bittersweetness
without retraining the model.
Usage
import torch
from transformers import AutoModel, AutoTokenizer
model_id = "lunahr/emotispace-128"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(
model_id,
trust_remote_code=True,
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()
@torch.no_grad()
def analyze(texts, top_k=5):
if isinstance(texts, str):
texts = [texts]
tokens = tokenizer(
texts,
return_tensors="pt",
padding=True,
truncation=True,
max_length=128,
).to(device)
out = model(**tokens)
embeddings = out["embeddings"].cpu()
probs = out["probs"].cpu()
results = []
for i, text in enumerate(texts):
top = torch.topk(probs[i], k=top_k)
results.append({
"text": text,
"embedding": embeddings[i],
"labels": [
{
"label": model.config.label_names[idx.item()],
"score": score.item(),
}
for score, idx in zip(top.values, top.indices)
],
})
return results
for item in analyze([
"I feel devastated.",
"This is amazing, I am so happy!",
]):
print(item["text"])
print(item["embedding"].shape)
print(item["labels"])
print()
Example behavior
Example output:
I feel devastated.
embedding shape: torch.Size([128])
sadness, remorse, disappointment, grief
This is amazing, I am so happy!
embedding shape: torch.Size([128])
joy, excitement, admiration, love
The embedding space can also separate emotional and non-emotional text. For example, emotionally charged text can be far away from factual neutral sentences like:
The table is made of wood.
Notes on embeddings
The 128D embedding is normalized and can be compared with cosine similarity.
Example use:
similarity = embedding_a @ embedding_b.T
The embedding space is designed to support emotional similarity, not just exact label matching.
For example:
"I feel devastated."
should be close to:
"I am so sad I could cry."
and far from:
"This is amazing, I am so happy!"
Limitations
This model is trained from English text and should be treated as English-first.
It is based on GoEmotions-style labels, so the classifier output is limited to that label space. The embedding space is more flexible, but custom emotion anchors should still be tested carefully.
The model does not actually “feel” emotions. It estimates emotional meaning from text.
Embeddings are useful for similarity and downstream control, but they are not a psychological diagnosis or a reliable mental health assessment tool.
Suggested downstream design
A recommended pattern is:
recent messages
-> EmotiSpace embeddings
-> weighted rolling mood embedding
-> compare against persona-agnostic emotion anchors
-> map to character response style or TTS controls
Keep emotion anchors persona-agnostic. Define what the emotion means generally, not how a specific character expresses it.
Good anchor style:
A calm emotional state with low tension, steady energy, and no urgency.
Avoid persona-specific anchors like:
Luna feels calm and wants to comfort you.
This prevents one character’s style from leaking into unrelated characters.
Citation
This model is based on BERT and trained using GoEmotions-style emotion supervision.
If you use this model, please credit:
lunahr/emotispace-128