Index-Echo-S2ST-2B-FP4-v41
Official NVFP4 (W4A4) quantization of IndexTeam/Index-Echo-S2ST-2B (v4.1), part of the Index-Echo speech-to-speech translation (S2ST) model family by bilibili.
The v4.1 package updates the multilingual translator and extends the release to twelve directions: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese.
This repository mirrors the original checkpoint layout; only the text LLM backbone (stlm_llm/) is quantized to NVFP4 - the audio tower, connector, and all speech-synthesis components remain in BF16. Use it exactly like the original repo (same modeling_dubbing.py / configs).
Quantization
- Scheme:
NVFP4(W4A4: 4-bit floating-point weights with per-group-16 scales, 4-bit floating-point activations with calibrated per-tensor global scales), produced with llm-compressor (quantization_schemerecorded inrecipe.yaml). Calibrated on a small bilingual translation corpus. - All
Linearlayers of the language model backbone (stlm_llm/) are quantized except the hybrid linear-attention projections; the audio tower, connector,lm_head, embeddings and all other pipeline components are kept in BF16. - Format: compressed-tensors
nvfp4-pack-quantizedsafetensors - load directly with vLLM (quantization="compressed-tensors") or transformers.
Consistency validation
Measured against the original BF16 checkpoint (greedy decoding, CPU reference execution with weight-dequantized linear layers):
| Metric | BF16 | FP4 | Delta |
|---|---|---|---|
| Perplexity (fixed corpus) | 10.1671 | 11.1610 | +9.78% |
| zh->en generation identical | - | - | yes |
| en->zh generation identical | - | - | no (semantically equivalent) |
Usage
Identical to the original checkpoint - clone this repo and follow the README / modeling_dubbing.py of the base model (IndexTeam/Index-Echo-S2ST-2B). The quantized LLM backbone loads via compressed-tensors; make sure compressed-tensors (or vLLM / a recent transformers) is installed.
Hardware note: full NVFP4 (W4A4) acceleration requires an NVIDIA Blackwell GPU (SM100+, e.g. B200 / RTX 50 series). On older GPUs vLLM falls back to weight-only dequantization, which still reduces memory but gives no FP4 speedup.
Quantized and published by the Index team, 2026-10-13.