whisper-base-khmer
A Khmer speech-to-text (STT) model — openai/whisper-base fine-tuned on Khmer speech. Built for iAny, the offline, on-device Khmer AI platform, and released open source so anyone can build Khmer voice tools.
Runs fully on-device / offline in the iAny app and mobile app. Part of iAny's mission: the best open Khmer AI, with the community and for the community.
What it does
Transcribes spoken Khmer audio (16 kHz mono) into Khmer text. Multilingual Whisper already knows Khmer; this fine-tune sharpens it on real Khmer corpora + community phone/room audio.
Formats in this repo
| Format | Files | For |
|---|---|---|
| PyTorch / Transformers | model.safetensors, config.json, tokenizer | training, server, research |
| ONNX | onnx/encoder_model.onnx, onnx/decoder_model_merged.onnx | browser / PWA (Transformers.js) |
| GGML | ggml-base-khmer-q5_1.bin | mobile / edge (whisper.cpp / whisper.rn) |
(CTranslate2 for faster-whisper can be produced with ct2-transformers-converter.)
Usage
🤗 Transformers
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="sengtha/whisper-base-khmer",
chunk_length_s=30, generate_kwargs={"language": "km", "task": "transcribe"})
print(asr("audio.wav")["text"])
Transformers.js (in the browser)
import { pipeline } from '@huggingface/transformers'
const asr = await pipeline('automatic-speech-recognition', 'sengtha/whisper-base-khmer', { dtype: 'fp32' })
const { text } = await asr(audioFloat32Array, { language: 'km', task: 'transcribe' })
whisper.cpp (GGML, on mobile/edge)
./whisper-cli -m ggml-base-khmer-q5_1.bin -l km -f audio.wav
Training data
- DDD-Cambodia/khmer-speech-dataset — large multi-speaker Khmer read speech (CC-BY-SA-4.0).
- sengtha/iany-khmer-voice — real phone/room audio contributed at iany.app/voice (CC-BY-SA-4.0), oversampled so the model weights real-world conditions. This set grows with community contributions, and the model is periodically retrained.
Evaluation
Character Error Rate (CER) is the meaningful metric for Khmer; report space-normalized CER since Khmer has no word spaces and Whisper inserts them. WER is not meaningful for Khmer (ambiguous word segmentation).
| Test set | CER (space-normalized) | CER (raw) |
|---|---|---|
FLEURS km_kh test (standard benchmark, out-of-domain read speech) | ~22.8 | ~23.9 |
| In-domain dev (training corpora) | ~19 | — |
FLEURS is clean, formal read speech with foreign proper nouns, so it's a harder, out-of-domain stress test — treat it as a comparability anchor, not a ceiling.
Limitations
- Best on clear, careful speech; spontaneous/noisy phone audio in a room is harder for a base-size model. More community
/voicedata + retraining is the path to better real-world accuracy. - Weak on numbers and embedded English/proper nouns.
- On low-confidence audio it can emit byte-fallback characters (rare); downstream apps may strip U+FFFD.
Intended use & responsible use
Voice input, dictation aids, transcription, and Khmer accessibility tools. It is an aid, not a certified transcription service — review output where accuracy matters.
License & attribution
Released under CC-BY-SA-4.0, inherited from the DDD-Cambodia training data. You must credit DDD-Cambodia and share derivatives alike. The base model openai/whisper-base is MIT.
Citation / credit
Trained and released by iAny (E-KHMER Technology). Thanks to DDD-Cambodia and every contributor at iany.app/voice. Training recipe: github.com/sengtha/iAny · docs/RUNPOD-KHMER-STT.md.