Nemotron-3.5-ASR-Streaming-0.6B GGUF

GGUF conversion of nvidia/nemotron-3.5-asr-streaming-0.6b for use with CrispASR.

Model details

Architecture: Cache-Aware Streaming FastConformer encoder (24 layers, d=1024, 8 heads) + RNN-T decoder (2-layer LSTM, hidden=640) + joint network (640 → 13088 vocab).

Languages: 39 languages, selected via prompt_kernel MLP conditioning (one-hot lang → 2048-dim hidden → 1024-dim encoder conditioning):

Key properties:

• Sample rate: 16 kHz mono
• 128 mel filterbank features, n_fft=512, hop=160 (10ms), win=400 (25ms)
• 8x time downsampling (causal) → 80ms frame duration
• Streaming: cache-aware attention with 4 context presets (see below)
• Vocab: 13087 SentencePiece tokens + 1 blank (pure RNN-T, no TDT durations)
• Native punctuation and capitalization
• License: OpenMDW-1.1 (permissive, commercial OK)

Files

Pre-encode weights are kept at F32 in both GGUFs (F16 causes 1.56 max accumulation error across the 4352-dim projection).

Usage with CrispASR

# Auto-download (Q4_K, 458 MB)
crispasr --backend nemotron -m auto --auto-download -f audio.wav

# Or download manually
huggingface-cli download cstr/nemotron-3.5-asr-streaming-GGUF \
  nemotron-3.5-asr-streaming-0.6b-q4_k.gguf --local-dir models/

# Transcribe (English, default)
crispasr --backend nemotron \
  -m models/nemotron-3.5-asr-streaming-0.6b-q4_k.gguf \
  -f audio.wav

# Transcribe in German
crispasr --backend nemotron \
  -m models/nemotron-3.5-asr-streaming-0.6b-q4_k.gguf \
  -f audio.wav -l de-DE

# Beam search (default is greedy)
crispasr --backend nemotron -m auto --auto-download -f audio.wav --beam-size 4

# Streaming from stdin
ffmpeg -i audio.wav -f s16le -ar 16000 -ac 1 - | \
  crispasr --backend nemotron -m auto --auto-download --stream

Streaming encoder

The model supports true cache-aware streaming via the NeMo cache_last_channel + cache_last_time architecture. Enable with:

CRISPASR_NEMOTRON_STREAMING=1 crispasr --backend nemotron -m model.gguf -f audio.wav

Four attention context presets trade latency for accuracy (published WER from NVIDIA's Open ASR Leaderboard):

Select preset: CRISPASR_NEMOTRON_CONTEXT_PRESET=3 (default: 0)

The same GGUF works for all presets — the context window is a runtime knob, not a retraining artifact.

Environment variables

Architecture

Audio (16kHz mono)
  → Mel spectrogram (128 bins, 10ms hop, no normalization)
  → Pre-encode (3x causal Conv2d, 8x downsample, Linear 4352→1024, F32 weights)
  → 24x Cache-Aware FastConformer block:
      FFN1(½) → MHA(rel_pos, cache-aware) → DWConv(k=9, causal, LN) → FFN2(½) → LN
  → Prompt kernel (MLP: concat(enc[1024], lang_onehot[128]) → 2048 → ReLU → 1024)
  → RNN-T decoder:
      Prediction: Embed(13088, 640) + 2-layer LSTM(640)
      Joint: enc(1024→640) + pred(640→640) → ReLU → Linear(640→13088)
  → Greedy / beam search decode

Streaming caches (per layer):

• cache_last_channel: post-FFN1 output (up to L frames), used as K/V context for asymmetric attention (Q from new frames only)
• cache_last_time: last K-1=8 frames of post-GLU signal before depthwise conv, prepended instead of zero-padding

Conversion

python models/convert-nemotron-to-gguf.py \
  --nemo nvidia/nemotron-3.5-asr-streaming-0.6b \
  --output nemotron-3.5-asr-streaming-0.6b-f16.gguf

crispasr-quantize nemotron-3.5-asr-streaming-0.6b-f16.gguf \
  nemotron-3.5-asr-streaming-0.6b-q4_k.gguf q4_k

Quality reference (JFK 11s)

F16 and Q4_K produce identical text. Streaming output has minor punctuation differences but same content.

Original model

• Source: nvidia/nemotron-3.5-asr-streaming-0.6b
• License: OpenMDW-1.1 — permissive, commercial use OK, derivatives OK with attribution
• Training data: 530k hours (NVIDIA Riva ASR set + Granary)