Parakeet TDT-CTC 0.6B (Japanese) — GGUF
GGUF / ggml conversions of nvidia/parakeet-tdt_ctc-0.6b-ja for use with the crispasr CLI from CrispStrobe/CrispASR.
A 600 M-parameter Japanese ASR model with punctuation:
- Hybrid FastConformer-TDT-CTC: TDT (Token-and-Duration Transducer) decoder by default; the CTC head is included in these GGUFs and selectable at runtime with
--parakeet-decoder ctc. - Built-in word-level timestamps from the TDT duration head — no separate CTC alignment.
- 6.4 % CER on JSUT basic5000.
- CC-BY-4.0 licence.
Files
All files include both the TDT decoder and the CTC head (added 2026-07;
earlier uploads lacked the CTC tensors, so --parakeet-decoder ctc silently
fell back to TDT).
| File | Size | Notes |
|---|---|---|
parakeet-tdt-0.6b-ja.gguf | 1.25 GB | F16, bit-exact match with NeMo on JSUT samples |
parakeet-tdt-0.6b-ja-q8_0.gguf | ~660 MB | Q8_0 — TDT output identical to F16 on our tests; recommended small file |
parakeet-tdt-0.6b-ja-q4_k.gguf | ~476 MB | Q4_K — TDT decode degrades on this model (repetition loops); use --parakeet-decoder ctc, which is clean at Q4_K. See note below |
Recommended: F16
Verified on a JSUT-basic5000 sample at F16:
NeMo (PyTorch): '水をマレーシアから買わなくてはならないのです。'
crispasr (F16): '水をマレーシアから買わなくてはならないのです。'
The F16 GGUF produces an identical transcript to the official NeMo Python pipeline.
About the Q4_K variant
The Japanese model uses an 80-mel preprocessor (vs. 128 for the
multilingual v3) and a smaller, more sensitive encoder distribution.
With our default Q4_K quantisation, two of the most logit-shaping
tensors (joint.pred.weight, decoder.embed.weight) fall back to
q4_0 because their dimensions don't tile cleanly for q4_k blocks.
This is enough quantisation noise that the TDT decoder enters a
fixed-point loop after the first ~8 tokens. Two clean options:
prefer the Q8_0 (TDT output identical to F16 in our tests), or
keep the Q4_K and decode with the CTC head
(--parakeet-decoder ctc) — CTC has no autoregressive feedback, so
the quantisation noise doesn't compound, and its output matches the
F16 CTC transcript.
Quick start
# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target crispasr-lib
# 2. Download the F16
huggingface-cli download cstr/parakeet-tdt-0.6b-ja-GGUF \
parakeet-tdt-0.6b-ja.gguf --local-dir .
# 3. Transcribe a 16 kHz mono WAV
./build/bin/crispasr --backend parakeet \
-m parakeet-tdt-0.6b-ja.gguf -f your-japanese-audio.wav -t 8
You can also let crispasr auto-download the model:
./build/bin/crispasr --backend parakeet -m auto --auto-download \
-f your-japanese-audio.wav --model-name parakeet-ja
Long-form audio (v0.8.8+)
The encoder is numerically fragile past ~12 s of context on real speech —
single-pass decoding of long audio silently drops content (upstream NeMo
behaves the same on the same clips: its plain, local-attention, and
buffered long-form modes score 1–51 % content recall on our reference
clip). CrispASR ≥ 3a8141e3 handles this automatically: audio > 30 s is
VAD-segmented, slices are capped at 12 s (split at energy minima), each
slice decodes in one NeMo-exact pass, and a gap-fill second pass
re-transcribes any span the first pass left empty. Measured on the
issue #89 reporter's clips (phonetic char-bigram recall vs
whisper-large-v3-turbo): 97.2 % (60 s), 96.9 % (120 s), 95.9 %
(300 s) — at the inter-model agreement ceiling (an independent
SenseVoice-small run scores the same recall on the same audio). No flags
needed; --vad, --chunk-seconds N, and CRISPASR_PARAKEET_* env vars
override the defaults (see the CrispASR CLI docs).
Word-level timestamps for free
Pass -v to dump per-token timestamps from the TDT duration head. Each token spans one or more encoder frames; one frame = 80 ms. No separate alignment model required.
Model architecture
| Component | Details |
|---|---|
| Encoder | 24-layer FastConformer, d=1024, 8 heads, head_dim=128, FFN=4096, conv kernel=9 |
| Encoder input | xscaling = True (input × √d_model = 32 before the first block) |
| Subsampling | Conv2d dw_striding stack, 8× temporal (50 → 12.5 fps) |
| Predictor | 2-layer LSTM, hidden 640, embed 3073 × 640 (blank padding-idx) |
| Joint head | enc(1024 → 640) + pred(640 → 640) → ReLU → linear(640 → 3078) |
| Vocab | 3072 SentencePiece tokens (Japanese, with punctuation) |
| Audio | 16 kHz mono, 80 mel bins, n_fft=512, hop=160, win=400 |
| Parameters | ~600 M |
xscaling=True is the most important architectural detail vs. the
multilingual v3 model — nvidia/parakeet-tdt-0.6b-v3 uses
xscaling=False. Both settings are stored in the GGUF metadata
(parakeet.xscaling) and read by the runtime, so the same code path
serves both variants without per-model branches.
How this was made
- The
.nemocheckpoint is unpacked; every architecture hyperparameter (d_model, n_layers, ff_dim, pred_hidden, joint_hidden, xscaling, …) is read frommodel_config.yamland cross-checked against the actual tensor shapes. The mel filterbank and Hann window are baked directly into the GGUF (preprocessor.fb,preprocessor.window). - NeMo state-dict keys are remapped to ggml-friendly names. Weights are
written as F16 for matmul tensors and F32 for norms / biases / mel
filterbank. A synthetic zero
conv.dw.biasis added per encoder layer when the checkpoint omits it (older NeMo BN-only convs). - Inference is implemented in
src/parakeet.{h,cpp}: the FastConformer encoder runs as a single ggml graph (BN folded into the depthwise conv weights at load time, xscaling applied between the pre-encode and the first block whenparakeet.xscaling=true), the LSTM predictor and joint head run as manual F32 CPU loops, and the TDT greedy decode loop alternates "advance encoder frame" / "emit token + advance predictor" using the duration head's argmax.
Attribution
- Original model:
nvidia/parakeet-tdt_ctc-0.6b-ja(CC-BY-4.0). NVIDIA NeMo team. - GGUF conversion + ggml runtime:
CrispStrobe/CrispASR.
Related
- Multilingual sibling (25 EU languages):
cstr/parakeet-tdt-0.6b-v3-GGUF - C++ runtime:
CrispStrobe/CrispASR
License
CC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.
Provenance and EU AI Act Art. 53 note
- Upstream model: nvidia/parakeet-tdt_ctc-0.6b-ja — published by
nvidia. - Upstream licence:
cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not. - What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
- Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
- Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.