Conformer Acoustic Landmark Detector (synthetic pretraining)
Frame-level detector of acoustic landmarks (Stevens, 2002) in speech audio: abrupt acoustic events tied to articulatory gestures. Given a waveform, the model outputs per-frame probabilities (10 ms resolution) for 8 landmark types plus background, from which landmark times are extracted by per-class peak picking.
This is the synthetic-pretrained variant: first pretrained on ~265k synthetic English words with automatically-detected landmarks (all-words-in-english-with-pink-trombone and vtl-speech-landmarks), then fine-tuned on the real manually-annotated corpus (1,839 recordings). The base variant without pretraining is available at conformer-landmark-detector. The pretraining-only checkpoint (before fine-tuning) is included under pretrain_only/.
Landmark types
| Type | Description | Precision | Recall | F1@20ms | Threshold |
|---|---|---|---|---|---|
| V | Vowel — peak of vowel/sonorant | 0.505 | 0.440 | 0.470 | 0.50 |
| G | Glide — vowel/consonant transition | 0.743 | 0.647 | 0.692 | 0.75 |
| Sc | Stop closure | 0.803 | 0.887 | 0.843 | 0.10 |
| Sr | Stop release burst | 0.854 | 0.886 | 0.870 | 0.40 |
| Fc | Fricative closure (frication onset) | 0.837 | 0.809 | 0.823 | 0.35 |
| Fr | Fricative release (frication offset) | 0.932 | 0.911 | 0.921 | 0.10 |
| Nc | Nasal closure (murmur onset) | 0.840 | 0.955 | 0.894 | 0.60 |
| Nr | Nasal release (murmur offset) | 0.786 | 0.733 | 0.759 | 0.60 |
Macro F1 @ 20 ms tolerance (held-out test set): 0.784
Precision/recall/F1 are landmark-level at ±20 ms tolerance on a held-out test split (10%, speaker-stratified). Detection thresholds were tuned per class on a separate validation split (10%) — the test set was never used for model selection or threshold tuning.
Architecture
Conformer encoder (12 blocks, d_model 256, 4 heads, FF 1024, conv kernel 31) over 80-band log-mel spectrograms (16 kHz, 25 ms window, 10 ms hop), with a linear classification head over 9 classes. ~18.3M parameters. Trained with Gaussian soft labels (per-class σ: V 20 ms, G 15 ms, Fc/Fr 12 ms, Sc/Sr/Nc/Nr 10 ms), class-weighted cross-entropy and label smoothing 0.1.
Usage
With the training repository (MateoCamara/conformer-landmark-detector):
pip install -r requirements.txt
python scripts/predict.py --audio path/to/audio.wav --checkpoint mcamara/conformer-landmark-detector-synthetic-pretrained
Or programmatically:
from src.utils.hub import load_landmark_model
model, config, thresholds = load_landmark_model("mcamara/conformer-landmark-detector-synthetic-pretrained", device="cuda")
# model: LandmarkClassifier over (batch, frames, 80) log-mel features
# config: mel front-end + label map + sigmas; thresholds: per-class operating points
The output JSON of predict.py contains {type, time_ms, confidence} entries.
Files
model.safetensors— model weightsconfig.json— architecture, mel front-end, label map, soft-label sigmasthresholds.json— per-class detection thresholds tuned on validationpretrain_only/model.safetensors— checkpoint after synthetic pretraining, before fine-tuning
Citation
@inproceedings{camara2026conformer-landmarks,
title={Conformer-based Acoustic Landmark Detection},
author={C\'amara, Mateo},
booktitle={Interspeech},
year={2026}
}
References
- Stevens, K. N. (2002). Toward a model for lexical access based on acoustic landmarks and distinctive features. JASA, 111(4), 1872-1891.
- Gulati, A., et al. (2020). Conformer: Convolution-augmented Transformer for Speech Recognition. Interspeech 2020.