Parakeet TDT 1.1B (en) — Transformers weights
A conversion of nvidia/parakeet-tdt-1.1b from the
NeMo .nemo format to 🤗 Transformers, produced with
src/transformers/models/parakeet/convert_nemo_to_hf.py. The weights are unchanged — only the checkpoint
format, the config and the tokenizer/feature-extractor wrappers differ.
FastConformer-XL encoder (42 layers, d_model=1024) with a TDT (Token-and-Duration Transducer) head,
durations [0, 1, 2, 3, 4], 1024 BPE tokens plus a blank token. Transcribes lower-case English without
punctuation.
Usage
import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, ParakeetForTDT
model_id = "extraordinarylab/parakeet-tdt-1.1b"
processor = AutoProcessor.from_pretrained(model_id)
model = ParakeetForTDT.from_pretrained(model_id, dtype=torch.float32).eval()
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
inputs = processor(ds[0]["audio"]["array"], sampling_rate=16000, return_tensors="pt")
outputs = model.generate(**inputs)
print(processor.batch_decode(outputs.sequences, skip_special_tokens=True))
Parity with NeMo
Checked against nemo_toolkit[asr] 3.0.0 on CPU in float32, on two LibriSpeech clips (13.7 s and 14.2 s),
both unbatched and as a padded batch of two:
| tensor | max abs diff |
|---|---|
| log-mel features | 6.5e-05 |
| encoder hidden states | 1.2e-04 |
joint logits (vocab + durations, at the blank SOS step) | 3.3e-04 |
Encoder output lengths agree (172 and 178 frames), and greedy decoding produces identical token ids and identical transcriptions in every configuration.
Notes on the conversion
Unlike parakeet-tdt-0.6b-v2/v3, this checkpoint's NeMo vocabulary carries no <pad> token, so — as for
the RNN-T checkpoints — <blank> takes the first free id (1024, matching NeMo's blank logit) and doubles
as the pad token. blank_token_id == pad_token_id == 1024, vocab_size == 1025, and the joint head emits
1025 + 5 logits.
License and attribution
Released under CC-BY-4.0, following the original model. The model was developed by the NVIDIA NeMo and Suno.ai teams; see the original model card for training data, evaluation results and limitations.