wav2vec2-ser-ravdess-optimized

facebook/wav2vec2-base-960h fine-tuned for 8-class speech emotion recognition on RAVDESS.

• Test accuracy: 68.3% (speaker-independent — evaluated on 3 actors never seen during training)
• Macro F1: 0.676
• Labels: angry, calm, disgust, fearful, happy, neutral, sad, surprised

Code and full writeup: https://github.com/sbhardwaj1304/Speech-Emotion-Recognition

Usage

import torch
from transformers import Wav2Vec2ForSequenceClassification, Wav2Vec2FeatureExtractor

MODEL_ID = "sbh013/wav2vec2-ser-ravdess-optimized"
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(MODEL_ID)
model = Wav2Vec2ForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()

import librosa
waveform, _ = librosa.load("speech.wav", sr=16000, mono=True)
inputs = feature_extractor(waveform, sampling_rate=16000, return_tensors="pt", padding=True)

with torch.no_grad():
    probs = torch.softmax(model(**inputs).logits, dim=-1).squeeze().numpy()

id2label = model.config.id2label
print({id2label[i]: float(probs[i]) for i in range(len(probs))})

Training details

Per-class results (test set)

Limitations

• Trained and evaluated only on RAVDESS (acted, English, studio-quality speech) — real-world microphone conditions and spontaneous (non-acted) speech may perform differently.
• sad is the weakest class, frequently confused with fearful and neutral.

Citation

• Livingstone, S. R., & Russo, F. A. (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). PLoS ONE, 13(5).
• Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020). wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. NeurIPS.