Conformer Acoustic Landmark Detector
Frame-level detector of acoustic landmarks (Stevens, 2002) in speech audio: abrupt acoustic events tied to articulatory gestures. Given a waveform, the model outputs per-frame probabilities (10 ms resolution) for 8 landmark types plus background, from which landmark times are extracted by per-class peak picking.
This is the base variant, trained only on the real manually-annotated corpus (1,839 recordings: 678 VCV syllables + 1,161 English words, 3 speakers). A variant pretrained on ~265k synthetic words is available at conformer-landmark-detector-synthetic-pretrained.
Landmark types
| Type | Description | Precision | Recall | F1@20ms | Threshold |
|---|---|---|---|---|---|
| V | Vowel — peak of vowel/sonorant | 0.545 | 0.532 | 0.538 | 0.45 |
| G | Glide — vowel/consonant transition | 0.743 | 0.647 | 0.692 | 0.75 |
| Sc | Stop closure | 0.826 | 0.849 | 0.837 | 0.30 |
| Sr | Stop release burst | 0.849 | 0.854 | 0.852 | 0.40 |
| Fc | Fricative closure (frication onset) | 0.833 | 0.787 | 0.809 | 0.55 |
| Fr | Fricative release (frication offset) | 0.867 | 0.867 | 0.867 | 0.50 |
| Nc | Nasal closure (murmur onset) | 0.824 | 0.955 | 0.884 | 0.65 |
| Nr | Nasal release (murmur offset) | 0.826 | 0.844 | 0.835 | 0.70 |
Macro F1 @ 20 ms tolerance (held-out test set): 0.789
Precision/recall/F1 are landmark-level at ±20 ms tolerance on a held-out test split (10%, speaker-stratified). Detection thresholds were tuned per class on a separate validation split (10%) — the test set was never used for model selection or threshold tuning.
Architecture
Conformer encoder (12 blocks, d_model 256, 4 heads, FF 1024, conv kernel 31) over 80-band log-mel spectrograms (16 kHz, 25 ms window, 10 ms hop), with a linear classification head over 9 classes. ~18.3M parameters. Trained with Gaussian soft labels (per-class σ: V 20 ms, G 15 ms, Fc/Fr 12 ms, Sc/Sr/Nc/Nr 10 ms), class-weighted cross-entropy and label smoothing 0.1.
Usage
With the training repository (MateoCamara/conformer-landmark-detector):
pip install -r requirements.txt
python scripts/predict.py --audio path/to/audio.wav --checkpoint mcamara/conformer-landmark-detector
Or programmatically:
from src.utils.hub import load_landmark_model
model, config, thresholds = load_landmark_model("mcamara/conformer-landmark-detector", device="cuda")
# model: LandmarkClassifier over (batch, frames, 80) log-mel features
# config: mel front-end + label map + sigmas; thresholds: per-class operating points
The output JSON of predict.py contains {type, time_ms, confidence} entries.
Files
model.safetensors— model weightsconfig.json— architecture, mel front-end, label map, soft-label sigmasthresholds.json— per-class detection thresholds tuned on validation
Citation
@inproceedings{camara2026conformer-landmarks,
title={Conformer-based Acoustic Landmark Detection},
author={C\'amara, Mateo},
booktitle={Interspeech},
year={2026}
}
References
- Stevens, K. N. (2002). Toward a model for lexical access based on acoustic landmarks and distinctive features. JASA, 111(4), 1872-1891.
- Gulati, A., et al. (2020). Conformer: Convolution-augmented Transformer for Speech Recognition. Interspeech 2020.