HuBERT Large German IPA Phoneme Scorer (v2)
Key Highlights & Performance
- Comprehensive Multi-Domain Training: Scaled from clean audiobooks to diverse acoustic environments including parliamentary debates, crowd-sourced phone audio, and studio recordings.
- Superior Accuracy: Reached an Evaluation Phoneme Error Rate (PER) of 22.28% (Accuracy > 77.7%) on unseen validation data, outperforming v1 across all acoustic categories.
- Low Latency: Average inference latency of ~32 ms per utterance on NVIDIA L4 (Tensor Cores / FP16), enabling real-time scoring in production.
📚 Training Dataset Architecture (123 GB Mega Dataset)
The model was fine-tuned on a composite dataset q1805/german-pronuncheck-mega-dataset, combining three complementary sources:
- 1.German Parliamentary Debates (Bundestag Corpus): Fast, spontaneous political discourse with natural room acoustics and public address microphones.
- 2.Mozilla Common Voice (German v17): Thousands of diverse speakers recorded on consumer smartphones and PC headsets with varied regional German dialects.
- 3.Multilingual LibriSpeech (MLS German): High-fidelity studio audiobook narrations.
- Total Samples: 1,249,116 training utterances + 86,575 validation utterances.
- Phonemizer: Converted to German IPA via
espeak-ngusing a fork-safe parallel pipeline (preserve_punctuation=False,with_stress=True).
⚙️ System-Level Engineering & Training Parameters
- Compute Infrastructure: Google Cloud Platform (GCP) Compute Engine VM.
- Accelerator: 1x NVIDIA L4 Tensor Core GPU (24 GB GDDR6 VRAM, Ada Lovelace architecture).
- Host Resources: 4 vCPUs, 16 GB RAM, 500 GB High-Throughput NVMe SSD.
- Total Training Time: 70 hours 25 minutes (73,050 total steps, 3 full epochs).
🛠️ Hardware & Memory Optimization Techniques:
- Mixed Precision (
fp16=True): Enabled NVIDIA L4 Tensor Cores, cutting memory by 50% and doubling matrix throughput. - Gradient Checkpointing (
gradient_checkpointing=True): Mitigated memory spikes from long audio sequences ($O(N^2)$ attention matrices). - Multi-Worker I/O (
dataloader_num_workers=4): Eliminated GPU starvation by asynchronously feeding audio arrays. - Effective Batch Size = 32: Configured via
per_device_train_batch_size = 2andgradient_accumulation_steps = 16. - Memory Defragmentation:
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. - Learning Rate Policy: Linear warmup (1,000 steps) to
3e-5, followed by linear decay to0.0.
📈 Comprehensive Benchmark Comparisons
1. Out-of-Domain Benchmark: Mozilla Common Voice Spontaneous Speech 4.0 (sps-corpus-4.0)
Evaluated on 100% of the completely unseen Mozilla Common Voice Spontaneous Speech 4.0 German dataset (natural speech with diverse accents and background noise):
| Benchmark Metric | Model_v1 | Model v2 (This repo) | Improvement |
|---|---|---|---|
| Phoneme Error Rate (PER) | 35.60% | 33.81% | -1.79% absolute error reduction 🎯 |
| Phonetic Accuracy | 64.40% | 66.19% | +1.79% 🚀 |
| Mean Inference Latency | 18.75 ms | 32.58 ms | Real-time ready |
| P50 Latency (Median) | 18.62 ms | 33.71 ms | Ultra-fast |
| P90 Latency | 19.30 ms | 35.03 ms | Smooth streaming |
| P99 Latency (Complex) | 23.14 ms | 41.44 ms | No bottlenecks |
2. Multi-Domain Benchmark: Mega Dataset 10% Held-Out Test Split (86,575 Utterances from q1805/german-pronuncheck-mega-dataset)
Evaluated on the independent 10% test split extracted from mega_dataset (comprising 86,575 utterances across Parliamentary debates, crowdsourced Mozilla audio, and MLS studio audio that neither model touched during training):
| Evaluation Metric | Model_v1 | Model_v2 (This repo) | Improvement |
|---|---|---|---|
| Phoneme Error Rate (PER) | 32.51% | 21.86% | -10.65% absolute error reduction 🎯 |
| Phonetic Accuracy | 67.49% | 78.14% | +10.65% 🚀 |
| Mean Inference Latency | 20.15 ms | 19.97 ms | Ultra-responsive |
| P50 Latency (Median) | 19.19 ms | 18.92 ms | Real-time ready |
| P90 Latency | 21.28 ms | 22.22 ms | Smooth streaming |
| P99 Latency (Complex) | 34.77 ms | 34.45 ms | No bottlenecks |
💻 Quickstart Inference Code
import torch
import librosa
from transformers import Wav2Vec2Processor, HubertForCTC
# Load production model and processor
REPO_ID = "q1805/hubert-german-IPA-large-v2"
processor = Wav2Vec2Processor.from_pretrained(REPO_ID)
model = HubertForCTC.from_pretrained(REPO_ID).eval()
# Load 16kHz audio
audio_path = "german_speech_sample.wav"
audio, sr = librosa.load(audio_path, sr=16000)
# Forward pass
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
logits = model(inputs.input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
ipa_transcription = processor.batch_decode(predicted_ids)[0]
print("Predicted German IPA:", ipa_transcription)