DuplexJev-32B-Turn

📄 Paper: arXiv:2610.02638 · Code: github.com/adventists-ai/duplexjev

DuplexJev-32B with a turn-taking upgrade. Same size, same serving stack, same prompt format; it decides much better when to speak, when to keep talking and when to stop, while spoken QA, gender and emotion stay where they were (or improve slightly).

DuplexJev reads typed, closed-set decisions from speech as a single-token distribution of the LLM: zero decode steps, many questions about the same clip in parallel. This repository is the complete model (Qwen3-ASR-0.6B encoder

• connector + the language model of Qwen3-VL-32B, 33.0 B parameters, bf16; one 80 GB GPU).

What changed

Starting from DuplexJev-32B we trained the connector together with a small LoRA on the LLM (rank 16 on q/k/v/o, 0.6 % of the parameters; merged into the weights here) on 110 k turn-taking questions rewritten from public corpora, mixed with the original content and paralinguistic data:

Most questions give the system state (the assistant is speaking / idle) and ask for an action (keep talking / stop and listen; reply / keep waiting); the rest ask for the user's turn state.

Results

Paper protocol (single-token readout, PyTorch), % correct. Through vLLM with duplexjev-vllm in the client wording: qa100 97, ZJU-ML 90, gender 92.4, emotion 90.5, Easy-Turn 95.6 (DuplexJev-32B: 94, 88, 89.8, 89.9, 79.4).

• Easy-Turn: 800-item four-way turn-state test set; the training mix contains the Easy-Turn training split (disjoint from the test set), so this score is in-domain.
• CoDeTT (arXiv 2603.25434): 18 k items, system state given, four actions averaged; zero-shot (no CoDeTT data in training). Our protocol: current utterance as audio, no history. For reference, published results: GPT-4o-audio 66.6 / 71.9, Qwen3-Omni 70.4 / 70.9, Gemini-3-Pro 80.8 / 81.9.
• † TurnBench-dev clips: our own clip-level protocol on the TurnBench development conversations (the official test labels are withheld): end of turn, pause inside a turn, backchannel, floor-taking interruption, noise / channel bleed; mean of four action accuracies. Not comparable with the official TurnBench leaderboard.
• Weakest remaining case: a pause in the middle of a long turn (the model takes the turn too early about half the time).

Serve with vLLM

pip install "vllm[audio]>=0.29" duplexjev-vllm
vllm serve adventists-ai/DuplexJev-32B-Turn --max-model-len 4096

Prompt format, client and decision tables are the same as for DuplexJev-32B; see examples/vllm_client.py. A state-aware question, for example:

The user said: <|audio|>

Question: The user said this while the assistant was speaking. What should the assistant do?

Options:
A. keep talking (backchannel, noise, or not addressed to the assistant)
B. stop and listen (interruption or a new request)

Answer with only the letter of the correct option.

Limitations

• Decides on one clip at a time; it is not a streaming end-of-turn detector by itself (call it at each pause).
• Pauses inside long turns are still often taken as the end of the turn.
• Emotion labels come from acted corpora; do not use the outputs to make decisions about individuals.

License

CC BY-NC 4.0, research use only (emotion data ESD is research-only; SmoothConv is CC BY-NC). Base models keep their licences: Qwen3-VL-32B-Instruct and Qwen3-ASR-0.6B are Apache-2.0 (Alibaba Cloud). The LLM weights here are the language model of Qwen3-VL-32B-Instruct with a LoRA merged in; the vision encoder is not included.

Citation

@misc{jin2026duplexjev,
  title         = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript},
  author        = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen},
  year          = {2026},
  eprint        = {2610.02638},
  archivePrefix = {arXiv},
  note          = {Submitted to IEEE ICASSP 2027}
}

Built by Adventists.ai. Claude (Anthropic) assisted with code.