RobotLegs STT ONNX models

This repository contains the independently downloadable runtime components for tenhaus/robotlegs-stt, converted from NVIDIA Canary-Qwen 2.5B at revision b1469e1bba1cfe140205529c79c434ca47180960 and Qwen3-1.7B at revision 0060bc56d46589041c1048efd1a397421b1142b5.

These files are intended for the Rust RobotLegs runtime. They are not a Transformers checkpoint.

Components

• encoder/: BF16 FastConformer audio encoder and projection into Qwen's embedding space.
• tokenizer/: the exact Qwen tokenizer used to encode the prompt and decode transcripts.
• qwen-bf16/: the evaluation BF16 Qwen decoder with the trained LoRA adapters merged.
• qwen-w4a4/: the production calibrated NVFP4 W4A4 Qwen decoder for Blackwell GPUs.
• manifests/: authenticated source, graph, runtime, and calibration contracts.
• distribution.json: immutable file sizes and SHA-256 identities for component downloads.

The audio frontend parameters are stored with the encoder. Prefill and cached decoding use the same Qwen graph; they are runtime phases, not separate models.

W4A16 is intentionally omitted. On the certification RTX 5090 it was 1.894x (89.4%) slower than BF16 and produced a larger complete deployment after its TensorRT engine and dependencies were counted.

RTX 5090 W4A4 result

The calibrated W4A4 graph quantizes the 196 Qwen attention and MLP projection linears while leaving the audio encoder, embedding/head, norms, rotary operations, logits, and KV cache in BF16.

For the pinned 13.69-second fixture, cached W4A4 inference had a 187.427 ms p50 versus 249.113 ms for BF16, a 1.329x (32.9%) speedup. Cached process memory was 4,582 MiB versus 7,964 MiB. The decoder compiled as one exclusive TensorRT partition, remained deterministic across 30 runs, and passed the exact 40-second input boundary.

This is hardware-specific evidence, not a general GPU performance claim. The benchmark fixture changed one phrase from schoolrooms under BF16 to school rooms under W4A4.

Official PyTorch/NeMo comparison

The pinned NVIDIA checkpoint was run through the documented eager SALM.generate path with PyTorch 2.13.0+cu130 and NeMo 3.1.0 at commit ba2cd63e. File decoding, model loading, and prompt tokenization were excluded from both timing boundaries; audio preprocessing, the encoder, prefill, greedy decoding, output-token transfer, and tokenizer decoding were included. Both profiles used batch size one, the same 13.69-second fixture, one warmup, and 30 measured runs.

RobotLegs delivered 2.236x (123.6%) higher single-utterance throughput, 55.3% lower p50 latency, and 20.1% less process GPU memory. Both profiles were deterministic across all measured runs. The official BF16 transcript used schoolrooms; W4A4 used school rooms.

The raw reports are pytorch-nemo-bf16-eager.json and w4a4-pytorch-comparison.json. This is a comparison with batch-one eager NeMo, not torch.compile, another PyTorch inference engine, batching, or NVIDIA's separately measured 418 RTFx result.

Earnings22 accuracy

The complete OpenASR Earnings22 test split was evaluated with greedy decoding. Both RobotLegs profiles transcribed all 2,741 rows; the same 2,737 non-empty references and 48,895 reference words as NVIDIA's published per-utterance result were scored with whisper-normalizer==0.1.12 and token-level Levenshtein alignment.

W4A4 adds 0.1145 percentage points (1.079% relative) to RobotLegs BF16. Both profiles stopped on EOS for every row. The two accuracy passes ran concurrently, so their row timings are not speed evidence. The full checked report, including artifact revisions, hashes, reference-matching evidence, and edit counts, is in benchmarks/earnings22-comparison.json.

Calibration

W4A4 used 512 real English clips: 256 LibriSpeech train-clean-100 utterances and 256 AMI Full-corpus-ASR training IHM oracle segments. The sorted calibration manifest SHA-256 is 8d6e21bbcb9e980ed3b92f46bec85a80e5ee122b7298ea81e3c31db77e5bd596.

Limitations

• English ASR only.
• RobotLegs currently accepts mono 16 kHz WAV/FLAC input of at most 40 seconds.
• The NVFP4 decoder is certified only on an RTX 5090 with the pinned ONNX Runtime 1.26, CUDA 13.3, and TensorRT 10.14.1 stack.
• The Hub Inference API and Transformers do not load these runtime-specific ONNX graphs directly.

License and attribution

The converted Canary-Qwen model is distributed under CC BY 4.0. Qwen3-1.7B is licensed under Apache 2.0. See LICENSES for the upstream notices and terms. This repository is an independently converted derivative and is not an official NVIDIA or Qwen release.