Zencastr Laughter Detection

Detect laughter events in close-microphone conversational speech. Built at Zencastr for podcast audio and open-sourced as a tool for creators.

Usage

Whole audio in, events out. Input can use any format ffmpeg can decode (path, URL, bytes, array) at any length or sample rate.

from transformers import pipeline

pipe = pipeline(
    "laughter-detection",
    model="zencastr/laughter-detection",
    trust_remote_code=True,
    device=0,
)

pipe("episode.mp3")
# {
#   "duration_s": 3591.4,
#   "events": [
#     {
#       "start": 812.16,
#       "end": 814.4,
#       "score": 0.997,
#     },
#     ...
#   ],
#   "operating_point": {
#     "name": "high_precision",
#     "threshold": 0.99,
#     "min_duration_s": 0.2,
#     "merge_gap_s": 1.0
#   },
# }

From the model repository, run the command-line scripts with uv:

uv run events.py episode.mp3 --operating-point max_f1 --pretty  # PyTorch
uv run events_onnx.py episode.mp3                               # ONNX

For clip-level classification of audio up to 4 s:

clf = pipeline(
    "audio-classification",
    model="zencastr/laughter-detection",
    trust_remote_code=True,
)
clf("clip.wav")
# [
#   {"label": "laughter", "score": 0.93},
#   {"label": "no_laughter", "score": 0.07}
# ]

Inference

Audio x is converted to a 128-bin log-mel spectrogram X at 16 kHz using 25 ms Hann windows and a 10 ms hop. The encoder f and laughter heads g then produce frame and clip probabilities. φ denotes the LoRA adapters. The model produces frame probabilities every 160 ms from 4 s windows evaluated with a 2 s hop. Scores from overlapping windows are combined by taking their maximum, then converted into events with approximate timestamps using a threshold, minimum duration, and merge gap. View the frame probabilities and tune the event parameters as follows:

pipe("episode.mp3", threshold=0.9, return_frame_probs=True)

The audio-classification pipeline instead returns one attention-pooled probability for the entire clip.

Operating points

Operating points are named presets for decoding frame probabilities into events.

pipe("episode.mp3", operating_point="max_f1")

The presets were selected on ~17 h of internal podcast audio from 21 recordings using a 0.2 s matching tolerance. high_precision maximizes recall at precision ≥ 0.88 and is recommended when false alarms are costly; high_recall reaches recall 0.88 and is recommended when missed laughter is costly; max_f1 maximizes F1 and provides a balance between precision and recall.

Evaluation

Our model achieves the highest mean clip-level average precision (AP) on three conversational benchmarks while maintaining a reasonable false positive rate (FPR) on other non-speech vocalizations.

Our row reports mean ± SD across multiple seeds while baselines rows report the performance of a single checkpoint ± the 95% bootstrap half-width. FSD50K FPR measures false positives on 1,136 clips of non-laughter vocalizations (coughing, sneezing, breathing, gasping, sighing, crying, and burping) at model-specific thresholds yielding 90% recall. CED, EAT, and BEATs use their AudioSet heads; SenseVoice uses its utterance-level laughter tag.

Inference speed

Real-time factors measured on an NVIDIA L4 using the same 1,800 s of audio, batch 64, and the median of nine passes. All models use fp16 and PyTorch except ours, which uses ONNX opset 23 with fused attention. Timings include windowing and host-to-device transfer, and exclude model loading, audio decoding, and event decoding.

Files

Provenance and licenses

Model and code: MIT (LICENSE). Base encoder: worstchan/EAT-base_epoch30_finetune_AS2M, revision 60d61e8b, weights SHA-256 5fa59385…69fff1b; MIT, © 2024 Wenxi Chen (LICENSE-EAT). The base encoder was pretrained and fine-tuned on AudioSet.

Adaptation used Zencastr platform audio collected under terms permitting model training. Public datasets were used only for evaluation and monitoring during adaptation. Dataset licenses: AudioSet annotations (CC BY 4.0), AMI (CC BY 4.0), VocalSound (CC BY-SA 4.0), PodcastFillers (CC BY-NC), and FSD50K (per-clip CC licenses).

Citation

@software{zencastr2026laughter,
  title  = {Zencastr Laughter Detection},
  author = {{Zencastr}},
  year   = {2026},
  url    = {https://huggingface.co/zencastr/laughter-detection}
}

@article{chen2024eat,
  title   = {EAT: Self-Supervised Pre-Training with Efficient Audio Transformer},
  author  = {Chen, Wenxi and Liang, Yuzhe and Ma, Ziyang and Zheng, Zhisheng and Chen, Xie},
  journal = {arXiv preprint arXiv:2401.03497},
  year    = {2024}
}