🇩🇪 HuBERT Large German IPA Phoneme Model (v1 - Baseline)
This repository hosts HuBERT Large German v1, an acoustic phoneme recognition model fine-tuned to predict the International Phonetic Alphabet (IPA) for German pronunciation evaluation.
📌 Model Overview
- Base Architecture:
facebook/hubert-large-ls960-ft(315M parameters). - Task: Connectionist Temporal Classification (CTC) Phoneme Recognition for German speech.
- Target Alphabet: 70 German IPA phoneme tokens (including vowels, diphthongs, consonants, stress markers
ˈ,ˌ, and German umlautsä,ö,ü,ß). - Objective: Serve as the baseline acoustic model for the PronunCheck pronunciation scoring system (Goodness of Pronunciation - GOP).
📚 Training Dataset
- Source: Multilingual LibriSpeech (MLS) - German.
- Size: approx. 20 GB of clean, studio-quality audiobook recordings (~160,000 utterances).
- Audio Characteristics: Clean studio narration, single speakers per chapter, 16 kHz sampling rate, minimal background noise.
- G2P Backend: Text labels were converted to German IPA using
espeak-ngviaphonemizerwith German dialect flags (de).
⚙️ Training Setup & Hyperparameters
- Hardware: 1x NVIDIA L4 GPU (24GB VRAM) on Google Cloud Platform (GCP).
- Precision: Mixed Precision (
fp16). - Batch Size: Per-device train batch size = 4, Gradient Accumulation = 4 (Effective batch size = 16).
- Optimizer: AdamW (
lr = 3e-5, linear warmup for 1,000 steps, linear decay). - Feature Extractor: Fixed 16 kHz CNN feature extractor (
freeze_feature_encoder=True). - Training Time: ~36 hours for 3 epochs.
📊 Evaluation & Limitations
- Clean Speech (MLS Test Split): PER ~28.4%.
- Spontaneous Speech (Mozilla SPS Corpus 4.0): PER ~35.60% (Accuracy: 64.40%).
- Known Limitations: Because v1 was trained purely on clean audiobook speech, its accuracy degrades noticeably when exposed to fast real-world speech, room reverberation, or low-grade microphones. (See v2 model for real-world robustness).
Quickstart Inference Code
import torch
import librosa
from transformers import Wav2Vec2Processor, HubertForCTC
# Load model and processor directly from Hugging Face
MODEL_ID = "q1805/hubert-large-german-v1"
processor = Wav2Vec2Processor.from_pretrained(MODEL_ID)
model = HubertForCTC.from_pretrained(MODEL_ID).eval()
# Load and resample audio
audio_path = "sample_german.wav"
speech, sr = librosa.load(audio_path, sr=16000)
# Inference
inputs = processor(speech, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
logits = model(inputs.input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
ipa_output = processor.batch_decode(predicted_ids)[0]
print("Predicted IPA:", ipa_output)