Audio CNN (ResNet-34) for Environmental Sound Classification (ESC-50)

A 21.3M-parameter ResNet-34 CNN that classifies 50 everyday sounds (dog bark, rain, siren, chirping birds, keyboard typing, …) from log-mel spectrograms. It was initialised from ImageNet weights and fine-tuned on ESC-50.

Code, training pipeline and an interactive layer-visualisation dashboard: github.com/Akash-Sakala/audio-cnn

Quick start

pip install torch torchaudio soundfile safetensors huggingface_hub

The architecture is plain PyTorch (no custom library), so download the repo and import model.py:

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("Akash-Sakala/audio-cnn-resnet34-esc50", allow_patterns=["*.py", "*.json", "*.safetensors"])
sys.path.insert(0, path)
from model import AudioClassifier

clf = AudioClassifier.from_pretrained(path)          # CPU or GPU, picked automatically
print(clf.predict("birds.wav", top_k=3))
# [('chirping_birds', 0.832), ('frog', 0.019), ('rain', 0.008)]

Or from the command line after cloning this repo:

python inference.py dog_bark.wav rain.wav --top-k 3

• Any input format: WAV, FLAC or OGG, at any sample rate, mono or stereo. Audio is converted to 44.1 kHz mono automatically.
• Feature maps: clf.predict(path, return_feature_maps=True) also returns the activations of every layer and residual block, for interpretability.

Model details

Stage output shapes (5 s clip): stem 64×32×108 → layer1 64×32×108 → layer2 128×16×54 → layer3 256×8×27 → layer4 512×4×14.

Input pipeline

All values are in config.json["frontend"]. model.py applies them automatically.

Training

Evaluation

Protocol.

• ESC-50's official 5-fold split is used; clips from the same source recording always stay in the same fold.
• Each model trains on 4 folds and is tested on the 5th.
• The reported number is the final-epoch EMA model. No checkpoint was chosen by looking at the test fold.

5-fold cross-validation (training recipe)

This checkpoint is the fold-4 model. Its 93.25% is the best of the five folds. Use 91.3% as the expected accuracy of the recipe. Same-seed reruns vary by about ±0.5 points because of GPU non-determinism.

This checkpoint on its held-out fold (400 clips, re-evaluated from model.safetensors)

35 of the 50 classes score 8/8. The errors are mostly between acoustically similar pairs:

Full per-class counts: eval/fold4_per_class.json. CV summary: eval/cv5_results.json.

Comparison on ESC-50

It is competitive with other ImageNet-pretrained CNNs. Models pretrained on AudioSet, a large audio dataset, still score higher.

Intended use and limitations

• Intended for: research, teaching and prototyping environmental-sound recognition, and studying CNN interpretability on spectrograms through the feature maps.

• Closed set of 50 classes. Any input is assigned one of the ESC-50 labels; there is no "unknown" class. Put a confidence threshold on top for open-world use.

• 5-second clips. Longer audio works, since pooling is global, but mixed-event recordings should be split into windows.

• Domain shift. ESC-50 clips are short, curated Freesound recordings. Accuracy on noisy real-world audio, other microphones or overlapping sources will be lower.

• Confidence scores are soft by design. Label smoothing and mixup keep correct predictions at a median top-1 probability of 0.84, while wrong ones have a median of 0.41 and a maximum of 0.72. That makes a simple threshold effective on held-out fold 4:

  Top-1 threshold	Coverage	Accuracy on kept clips
  none	100%	93.3%
  ≥ 0.5	84.8%	97.3%
  ≥ 0.6	81.2%	98.5%

• Non-commercial. The model inherits ESC-50's CC BY-NC 3.0 licence, so it is not for commercial use.

Files

Reproduce

git clone https://github.com/Akash-Sakala/audio-cnn && cd audio-cnn
pip install -r requirements-local.txt && modal setup
modal run train_v2.py --folds 1,2,3,4,5 --run-name v2_cv5

Licence and citation

• Weights: CC BY-NC 3.0, following the ESC-50 dataset licence.
• Code (GitHub): MIT. It builds on Andreas Trolle's audio-cnn.

@inproceedings{piczak2015dataset,
  title     = {{ESC}: {Dataset} for {Environmental Sound Classification}},
  author    = {Piczak, Karol J.},
  booktitle = {Proceedings of the 23rd {Annual ACM Conference} on {Multimedia}},
  pages     = {1015--1018},
  year      = {2015},
  doi       = {10.1145/2733373.2806390}
}

Author: Akash S S · GitHub · LinkedIn