whisper-large-v3-turbo tokenizer with <|startofprev|> renamed to <|prev|>
A drop-in tokenizer-only workaround for
vllm-project/vllm#53206:
passing the OpenAI-compatible prompt field to vLLM's
/v1/audio/transcriptions on a Whisper model corrupts decoding — degraded
output on short audio, and on audio longer than 30 s the response silently
collapses to only the final chunk's text.
No model weights here. These are the official
openai/whisper-large-v3-turbo
tokenizer files with exactly one change: the string form of token id 50362
is renamed from <|startofprev|> to <|prev|> (applied consistently in
tokenizer.json, tokenizer_config.json, added_tokens.json and
special_tokens_map.json). Vocabulary size and every id are unchanged.
The repo also carries the unmodified official config.json: vLLM
resolves a remote --tokenizer repo through AutoConfig and refuses to
start when the repo lacks a config.json with a model_type key (local
directory mounts skip that check). No weights are read from this repo.
Why this works
vLLM builds the Whisper decoder prompt as:
decoder_text = (
f"<|prev|>{request_prompt}" if request_prompt else ""
) + f"<|startoftranscript|><|{language}|><|{task_type}|><|notimestamps|>"
<|prev|> is not in the Whisper vocabulary, so the stock tokenizer splits
it into six ordinary text tokens (<, |, pre, v, |, >) placed in
front of <|startoftranscript|>, which corrupts decoder conditioning. The
real prompt-conditioning token is <|startofprev|> (id 50362).
With this tokenizer mounted, the same <|prev|> string tokenizes straight to
id 50362 — the genuine sot_prev embedding — so the prompt field behaves
as intended again, with zero code changes to vLLM. Requests without a
prompt are byte-for-byte unaffected.
Usage
vllm serve openai/whisper-large-v3-turbo \
--tokenizer ardge-hf/whisper-large-v3-turbo-tokenizer-prevfix
Then the standard OpenAI prompt field works:
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.mp3 \
-F response_format=verbose_json \
--form-string 'prompt=這是繁體中文的語音轉錄。'
Measured on vllm/vllm-openai:nightly (v0.26.1rc1, 2026-08): without the fix
the request above returns a single hallucinated segment; with it, the full
transcript comes back and the prompt bias takes effect (e.g. Mandarin output
switches from Simplified to Traditional script with punctuation).
Verify
from transformers import AutoTokenizer
t = AutoTokenizer.from_pretrained("ardge-hf/whisper-large-v3-turbo-tokenizer-prevfix")
assert t.encode("<|prev|>", add_special_tokens=False) == [50362]
assert len(t) == 51866 # no phantom tokens added
When to stop using this
The proper fix — vLLM emitting <|startofprev|> in the first place — is
vllm-project/vllm#53207.
Once your vLLM build contains it, drop the --tokenizer flag and go back to
the stock tokenizer.
Scope and caveats
- Built for openai/whisper-large-v3-turbo and valid for the v3 family,
where
sot_previs id 50362. The v2 family (large-v2, small, …) uses id 50361 — regenerate per checkpoint rather than reusing this one. - Client-side workarounds do not exist for this bug: the junk
<|prev|>text tokens are prepended server-side before any client-controlled content, and even embedding a literal<|startofprev|>inside thepromptvalue still collapses (measured). The tokenizer has to be swapped server-side. - Decoding id 50362 now renders
<|prev|>; withskip_special_tokens=True(the transcription default) this is never visible. - Serving/inference workaround only — do not fine-tune against this tokenizer.
Provenance
Tokenizer files come from
openai/whisper-large-v3-turbo
(Apache-2.0), modified as described above. The rename was verified against
the official added_tokens_decoder tables of
whisper-large-v3-turbo
and
whisper-large-v3,
and the reference specials list in
openai/whisper tokenizer.py.