Whisper-Base for PERCEPT-R Classification (Audio classification of /ɹ/ in children)
Model Description
This model includes the implementation for audio classification of /ɹ/ in children described in ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood (Accepted to EMNLP 2026 Main)
Github repository: https://github.com/tiantiaf0627/childvox-release
The model is fine-tuned on the PERCEPT-R dataset, a large-scale corpus for audio classification of /ɹ/ in children.
The included categories are:
[
'Derhotic',
'Rhotic'
]
Canonical denotes mature syllables containing a consonant-vowel transition, while Non-Canonical denotes immature vocalizations such as isolated vowels or consonants. Junk covers segments that are not child vocalizations (e.g., noise, adult speech, or unintelligible audio).
How to use this model
Download repo
git clone git@github.com:tiantiaf0627/childvox-release
Install the package
conda create -n childvox python=3.10
cd childvox
pip install -e .
Load the model
# Load libraries
import torch
import torch.nn.functional as F
from src.model.childvox.whisper_audio import WhisperWrapper
# Find device
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"
# Load model from Huggingface
# We provide model with different folds, and specify the fold from 1, 2, 3, 4, 5
model = WhisperWrapper.from_pretrained("tiantiaf/childvox-percet_r-whisper-base", fold_idx=1).to(device)
model.eval()
Prediction
# Label List
label_list = [
'Derhotic',
'Rhotic'
]
# Load data, here just zeros as the example
# The child word reading segments used in training are short, so we cap the input at 2 seconds
# You need to prepare your audio to a length of 2 seconds, 16kHz and mono channel
max_audio_length = 2 * 16000
data = torch.zeros([1, 160000]).float().to(device)[:, :max_audio_length]
logits, embeddings = model(data, return_feature=True)
# Probability and output
r_prob = F.softmax(logits, dim=1)
print(label_list[torch.argmax(r_prob).detach().cpu().item()])
Responsible Use: Child speech data is highly sensitive. Users should respect the privacy and consent of the children and families whose recordings are processed, obtain approval from the appropriate ethics/IRB body, and adhere to the relevant laws and regulations in their jurisdictions when using ChildVox.
If you have any questions, please contact: Tiantian Feng (tiantiaf@usc.edu)
❌ Out-of-Scope Use
- Clinical or diagnostic applications (e.g., screening for developmental or language disorders)
- Individual-level developmental assessment without expert human review
- Surveillance
- Privacy-invasive applications
- No commercial use
If you like our work or use the models in your work, kindly cite the following. We appreciate your recognition!
@article{feng2026childvox,
title={ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood},
author={Feng, Tiantian and Xu, Anfeng and Shi, Xuan and Kommineni, Aditya and Siam, Shakhrul Iman and Micheletti, Megan and Shi, Zhonghao and Tager-Flusberg, Helen and Zhang, Mi and Perry, Lynn K and others},
journal={arXiv preprint arXiv:2605.29257},
year={2026}
}
You will also need to cite the following dataset paper and practice if you want to use this model
The model us derived from data hosted on PhonBank, part of the TalkBank system. It is subject to TalkBank's Ground Rules and distributed under CC BY-NC-SA 3.0.
@inproceedings{benway22_interspeech,
title = {{PERCEPT-R: An Open-Access American English Child/Clinical Speech Corpus Specialized for the Audio Classification of /ɹ/}},
author = {Nina Benway and Jonathan L. Preston and Elaine Hitchcock and Asif Salekin and Harshit Sharma and Tara McAllister},
year = {2022},
booktitle = {{Interspeech 2022}},
pages = {3648--3652},
doi = {10.21437/Interspeech.2022-10785},
issn = {2958-1796},
}