Model Card for whisper-large-v2-taiwanese-hakka-v1

This model is a fine-tuned version of the Taiwanese hakka openai/whisper-large-v2.

Training process

The training of the model was performed with the following hyperparameters

• Batch size: 2*4 (on 4 L40s GPU)
• Gradient accumulation steps: 64
• Total steps: 3828
• Learning rate: 1e-4
• Data augmentation: No
• Optimizer: schedule_free_radam
• LR scheduler type: constant

How to use

Access and Authentication

This model is hosted as a gated Hugging Face repository. Before using it:

1. Visit the model page and request access.
2. Log in with the same Hugging Face account that has been granted access.
3. Authenticate your local environment with a Hugging Face access token.

A read token is sufficient for inference.

pip install -U huggingface_hub
hf auth login

Alternatively, you can provide the token through the HF_TOKEN environment variable:

export HF_TOKEN=hf_xxx

Do not hard-code your Hugging Face token in scripts, notebooks, or public repositories.

If you see an error such as Cannot access gated repo, make sure that:

• your Hugging Face account has been granted access to this model;
• hf auth whoami shows the expected account;
• HF_HUB_DISABLE_IMPLICIT_TOKEN is not set.

Run Model

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
model_id = "formospeech/whisper-large-v2-taiwanese-hakka-v1"
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)
processor = AutoProcessor.from_pretrained(model_id)
pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    chunk_length_s=30,
    batch_size=16,
    torch_dtype=torch_dtype,
    device=device,
)
generate_kwargs = {"language": "id"}
transcription = pipe("path/to/my_audio.wav", generate_kwargs=generate_kwargs)
print(transcription)