Whisper Medium ATC full
This model is a fine-tuned openai/whisper-medium on Czech and English air traffic communication recordings from Czech airport LKKU.
It was created as a product of bachelor's thesis at Faculty of Information Technology Brno University of Technology.
Model description
- Developed by: Veronika Nevarilova (@xnevar00), Igor Szoke (@iszoke)
- Shared by: BUT FIT
- Model type: Whisper
- Languages: Czech, English
- License: Apache 2.0
- Finetuned from model: openai/whisper-medium
Usage
import torch
from transformers import pipeline
audio = "path/to/audio.xx"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
transcribe = pipeline(task="automatic-speech-recognition", model="BUT-FIT/whisper-ATC-czech-full", chunk_length_s=30, device=device)
transcribe.model.config.forced_decoder_ids = transcribe.tokenizer.get_decoder_prompt_ids(task="transcribe", language="czech")
print('Transcription:', transcribe(audio)["text"])
Dataset
Training dataset was made of ~5 hours of air traffic communication recordings. Recordings were Czech and English (80:20) and sporadically Slovak.
Output format
The model was learned to transcribe every recording word by word. Transcription format of a recording is as follows:
Recording: Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů
Transcription: Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů
Note: See also model BUT-FIT/whisper-ATC-czech-short, which abbreviates callsigns and numbers.
Results
The model reached total WER of 14.7 % on unseen Czech and English LKKU recordings. 19.6 % WER was achieved on a testset containing Czech air traffic recordings from other airports, LKPR and LKTB.
WER of callsings in LKKU recordings was evaluated to be 6.2 %, while on LKPR and LKTB dataset the model reached 3.6 %.
Training hyperparameters
- learning_rate: 3e-5
- per_device_train_batch_size: 2
- gradient_accumulation_steps: 8
- warmup_ratio: 0.12
- fp16: True
- gradient_checkpointing: True
- evaluation_strategy: "epoch"
- save_strategy: "epoch"
- load_best_model_at_end: True
- metric_for_best_model: "wer"
- num_train_epochs: 45
Contact
For further information don't hesitate to contact Veronika Nevarilova (xnevar00@stud.fit.vutbr.cz) or Igor Szoke (szoke@fit.vutbr.cz).