distilhubert-finetuned-Donate_a_cry-v3

This model is a fine-tuned version of ntu-spml/distilhubert on the Nooon/Donate_a_cry dataset for baby cry classification — detecting the reason behind an infant's cry from raw audio.

Model Description

This model classifies baby cry audio recordings into 5 categories representing common infant distress signals:

The model uses DistilHuBERT, a distilled version of HuBERT (Hidden-Unit BERT), which is a self-supervised speech representation model. It was fine-tuned end-to-end on balanced baby cry audio data to learn acoustic patterns associated with each cry type.

Intended Uses & Limitations

✅ Intended Uses

• Assisting parents and caregivers in understanding the cause of infant crying
• Integration into baby monitor applications or smart home systems
• Academic research in infant vocal expression and affective computing
• Part of a broader infant health monitoring pipeline (e.g., combined with postpartum depression detection)

⚠️ Limitations

• The model was trained on a single dataset (Donate_a_cry), which may not cover all acoustic environments, microphone types, or infant demographics
• Performance may degrade with background noise or low-quality microphone recordings
• The model is not a medical device and should not replace professional pediatric advice
• Very short clips (under 1–2 seconds) or clips longer than 10 seconds may produce less reliable predictions
• The model was trained on 16 kHz audio; input at different sampling rates will be automatically resampled by the feature extractor

Training and Evaluation Data

Dataset: Nooon/Donate_a_cry

The Donate_a_cry dataset contains real infant cry recordings labeled across 5 categories. The original dataset has class imbalance, which was addressed before training.

Preprocessing & Balancing Strategy

The raw dataset was preprocessed and balanced using the following pipeline:

1. Padding / Truncation — All audio arrays were normalized to a fixed length of 16,000 × 10 = 160,000 samples (10 seconds at 16 kHz) using zero-padding for shorter clips and truncation for longer ones.
2. SMOTE (Synthetic Minority Over-sampling Technique) — Applied to the flattened audio feature arrays to generate synthetic samples for under-represented classes, producing a fully balanced training set.
3. Train/Test Split — The balanced dataset was split into 80% training / 20% evaluation using a fixed seed of 42 for reproducibility.
4. Feature Extraction — The AutoFeatureExtractor from DistilHuBERT was applied with:
   • Sampling rate: 16,000 Hz
   • Max duration: 10 seconds
   • Normalization: enabled
   • Attention mask: returned
   • Dropout on features, hidden states, and projections for regularization

Training Procedure

Architecture

• Base model: ntu-spml/distilhubert
• Classification head: Linear layer with 5 output classes
• Dropout settings applied at feature extraction:
  • attention_dropout = 0.5
  • hidden_dropout = 0.5
  • final_dropout = 0.5
  • feat_proj_dropout = 0.5
  • mask_time_prob = 0.05 (SpecAugment-style masking)

Training Hyperparameters

Regularization & Stability

• Early stopping with patience of 3 epochs was applied to prevent overfitting
• Model checkpoints were saved each epoch; the best checkpoint (by eval accuracy) was loaded at the end

Framework Versions

Evaluation

The model was evaluated on a held-out 20% split of the SMOTE-balanced dataset. The following metrics were computed:

• Accuracy — overall correctness across all 5 classes
• Weighted F1-score — harmonic mean of precision and recall, weighted by class support
• Weighted Precision — weighted average precision per class
• Weighted Recall — weighted average recall per class
• Confusion Matrix — per-class breakdown of predictions vs. true labels
• ROC Curves — one-vs-rest AUC for each of the 5 classes

How to Use

Using the Pipeline API (recommended)

from transformers import pipeline

pipe = pipeline("audio-classification", model="AmeerHesham/distilhubert-finetuned-baby_cry")

# Pass a local audio file or URL
result = pipe("baby_cry_sample.wav")
print(result)
# Example output: [{'score': 0.82, 'label': 'hungry'}, ...]

Loading the Model Directly

from transformers import AutoProcessor, AutoModelForAudioClassification
import torch

processor = AutoProcessor.from_pretrained("AmeerHesham/distilhubert-finetuned-baby_cry")
model = AutoModelForAudioClassification.from_pretrained("AmeerHesham/distilhubert-finetuned-baby_cry")

# Load your audio (must be 16kHz)
import librosa
audio, sr = librosa.load("baby_cry_sample.wav", sr=16000)

inputs = processor(audio, sampling_rate=sr, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
print(model.config.id2label[predicted_class_id])

Label Mapping

id2label = {
    '0': 'belly_pain',
    '1': 'burping',
    '2': 'discomfort',
    '3': 'hungry',
    '4': 'tired'
}

Citation

If you use this model in your research or application, please cite the base model and dataset:

@misc{distilhubert-baby-cry,
  author    = {Amir Hesham},
  title     = {DistilHuBERT Fine-tuned for Baby Cry Classification},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/AmeerHesham/distilhubert-finetuned-baby_cry}
}

Related Work

This model was developed as part of the Bambino graduation project — a mobile application for pregnant mothers and infants that integrates:

• 🍼 Baby cry classification (this model) — detecting infant needs from audio
• 🧠 Postpartum depression detection — supporting maternal mental health monitoring

Model developed by AmeerHesham