Sagarmatha V4 - Nepali Automatic Speech Recognition
Sagarmatha V4 is a fine-tuned version of openai/whisper-large-v3-turbo for Nepali (Devanagari script) automatic speech recognition. It was trained on 265 hours of curated Nepali audio using parameter-efficient fine-tuning (QLoRA) and achieves a Word Error Rate of 30.99% on a held-out test set, representing approximately a 20 percentage point improvement over the zero-shot baseline.
Model Details
Model Description
| Attribute | Value |
|---|---|
| Model type | Encoder-decoder (Whisper architecture) |
| Base model | openai/whisper-large-v3-turbo |
| Fine-tuning method | QLoRA (LoRA rank 32, alpha 64, NF4 quantization) |
| Language | Nepali (ne) |
| License | Apache 2.0 |
| Release date | July 2026 |
This repository contains the fully merged model: the LoRA adapter weights have been folded into the base model parameters. No PEFT library is required at inference time.
Intended Use
- Transcription of spoken Nepali audio to Devanagari text
- Research on low-resource South Asian language speech recognition
- Downstream NLP pipelines requiring Nepali ASR
Out-of-Scope Use
- Translation tasks (the model was fine-tuned for transcription only)
- Languages other than Nepali
- Real-time streaming transcription without chunking for audio longer than 30 seconds
Evaluation Results
Evaluated on held-out splits of the Sagarmatha V4 dataset after 10,000 training steps (~3.5 epochs). Two splits were used: a random sample across all clip lengths, and a long-clip split restricted to utterances of more than 25 words.
| Evaluation Split | WER | CER |
|---|---|---|
| Random sample (all lengths) | 30.99% | 9.58% |
| Long clips (>25 words) | 31.70% | 10.20% |
| Baseline: zero-shot whisper-large-v3-turbo | ~52% | - |
The near-identical WER between short and long clips confirms that the EOS truncation bias present in earlier model versions (Sagarmatha V3, WER 55.74% with 44% deletions on long clips) has been fully resolved in this version.
Usage
Direct Inference
from transformers import pipeline
pipe = pipeline(
"automatic-speech-recognition",
model="tonibirat/sagarmatha-v4-nepali-asr",
generate_kwargs={"language": "nepali", "task": "transcribe"},
device=0, # use -1 for CPU
)
result = pipe("audio.wav")
print(result["text"])
Long-Form Audio (recommended for audio longer than 30 seconds)
result = pipe(
"long_audio.wav",
return_timestamps=True,
chunk_length_s=30,
stride_length_s=5,
)
print(result["text"])
for chunk in result["chunks"]:
start, end = chunk["timestamp"]
print(f"[{start:.1f}s - {end:.1f}s] {chunk['text']}")
With WhisperProcessor Directly
import torch
from transformers import AutoProcessor, WhisperForConditionalGeneration
processor = AutoProcessor.from_pretrained("tonibirat/sagarmatha-v4-nepali-asr")
model = WhisperForConditionalGeneration.from_pretrained(
"tonibirat/sagarmatha-v4-nepali-asr",
torch_dtype=torch.float16,
device_map="auto",
)
# Prepare audio (16 kHz mono expected)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(
inputs["input_features"].to(model.device, dtype=torch.float16),
language="nepali",
task="transcribe",
)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])
Training Details
Dataset
The model was trained on Sagarmatha V4, a curated corpus of 265 hours of Nepali speech comprising 156,375 audio clips in FLAC format.
| Attribute | Value |
|---|---|
| Total duration | 265 hours |
| Number of clips | 156,375 |
| Audio format | FLAC, 16 kHz mono |
| Sources | OpenSLR-54 (Nepali speech), Internal curated v3 |
| Quality filter | CTC confidence score >= 0.70 |
| Evaluation split | Random held-out set + long-clip set (>25 words) |
Data from the OpenSLR source was obtained from openslr.org/54 (Nepali TTS and ASR corpus).
Hyperparameters
| Parameter | Value |
|---|---|
| Base model | openai/whisper-large-v3-turbo |
| Fine-tuning method | QLoRA |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, out_proj, fc1, fc2 |
| Quantization | NF4 (4-bit) |
| Training steps | 10,000 |
| Effective epochs | ~3.5 |
| Effective batch size | 32 (per-device batch 2, gradient accumulation 16) |
| Learning rate | 1e-5 |
| LR schedule | Linear decay with 1,000 warmup steps |
| Optimizer | AdamW |
| Final training loss | 0.5849 |
Hardware
| Attribute | Value |
|---|---|
| Hardware | 2x NVIDIA Tesla T4 (Kaggle, 16 GB VRAM each) |
| Training duration | ~10.3 hours |
| Framework | Hugging Face Transformers 4.x + PEFT |
| Precision | Mixed precision (fp16) |
Limitations
- Accent coverage: The model was trained predominantly on standard spoken Nepali. Performance on strong regional accents or dialects has not been evaluated.
- Code-switching: Utterances mixing Nepali and English may produce degraded output, as the training corpus does not contain code-switched speech.
- Noisy environments: No noise augmentation was applied during training. Performance in high-noise conditions is expected to be lower.
- Long-form audio: Clips longer than 30 seconds require chunked inference. End-to-end transcription of arbitrarily long audio without chunking is not supported.
- Translation: The model was not fine-tuned for translation. Invoking
task="translate"will produce Whisper's default translation behaviour, not a fine-tuned translation system.
Citation
If you use this model in your research, please cite the following:
@misc{birat2026sagarmatha,
author = {Birat, Toni},
title = {Sagarmatha V4: QLoRA Fine-Tuning of Whisper for Low-Resource Nepali ASR},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/tonibirat/sagarmatha-v4-nepali-asr}},
}
Acknowledgements
Base model weights are from openai/whisper-large-v3-turbo. Training was conducted on Kaggle (free GPU tier, T4 hardware) using the Hugging Face PEFT and Transformers libraries.