MiniCPM-duplex-rl
A full-duplex turn-taking RL fine-tune of
enochlev/MiniCPM-duplex
(MiniCPM-duplex, from xinrongzhang2022/MiniCPM-duplex). The model decides every
~1.7 s block whether to speak or stay silent while the user may also be speaking;
this checkpoint was trained with REINFORCE over block-level turn-taking rewards
(interruption penalties, timely-response rewards, silence penalties) for 180 steps.
Effect: compared to the base model it interrupts the user less, yields to barge-ins, and resumes after overlapping speech — trading away some take-turn responsiveness on direct interruptions.
Training + serving code: enochlev/text-only-duplex-model
Serving
Serve bf16 (fp8 + greedy sampling breaks the idle/speak decision):
vllm serve enochlev/MiniCPM-duplex-rl \
--served-model-name cpm-text-duplex --max-model-len 3000 \
--gpu_memory_utilization 0.30 --trust-remote-code
then point the repo's server.py --cpm at it for the real-time audio stack
(Kokoro TTS + Parakeet ASR + WebSocket client protocol).
FullDuplexBench results (base vs this model)
Evaluated with Full-Duplex-Bench (GPT-4o behavior classification).
v1.5 — behavior distribution + stop/response latency (pooled, seconds)
| Task (desired) | Model | n | RESPOND | RESUME | Stop (s) | Resp (s) |
|---|---|---|---|---|---|---|
| user_interruption (RESPOND ↑) | base | 200 | 0.65 | 0.20 | 2.17 | 1.93 |
| rl | 175 | 0.49 | 0.36 | 2.21 | 2.60 | |
| user_backchannel (RESUME ↑) | base | 98 | 0.00 | 0.52 | 0.73 | 1.93 |
| rl | 98 | 0.00 | 0.63 | 0.67 | 2.03 | |
| talking_to_other (RESUME ↑) | base | 100 | 0.47 | 0.24 | 1.43 | 1.90 |
| rl | 100 | 0.28 | 0.43 | 1.53 | 2.18 | |
| background_speech (RESUME ↑) | base | 100 | 0.63 | 0.25 | 1.21 | 2.27 |
| rl | 98 | 0.45 | 0.31 | 1.19 | 2.32 |
The RL model wins the three tasks whose desired behavior is staying quiet / resuming (backchannels, third-party speech, background speech) and is less eager on direct user interruptions.
v1.0 — turn-taking dimensions
| Metric | base | rl |
|---|---|---|
| Candor Pause Handling · take-turn | 0.916 | 0.635 |
| Candor Turn Taking · take-turn / latency | 0.992 / 0.31s | 0.861 / 0.85s |
| ICC Backchannel · JSD / TOR / Freq | 0.44 / 0.71 / 0.44 | 0.69 / 0.73 / 0.15 |
| Synthetic Pause Handling · take-turn | 0.934 | 0.653 |
| Synthetic User Interruption · rating / take-turn / latency | 4.15 / 1.0 / 0.71s | 4.04 / 0.98 / 1.76s |
v1.0's take-turn/latency conventions favor the eager base model; the consistent direction across both versions reflects the RL objective — restraint over eagerness.
Training summary
- 180 REINFORCE steps, lr 5e-6, 32 episodes/step, γ=0.90, per-batch z-scored advantages
- Block-level rewards: interruption penalty, timely-response reward, silence penalty, missed-turn penalty, backchannel-loop penalty
- Seed-reproducible (two independent seeds: best avg reward +1.36 / +1.35); replay eval cut stale-overlap speech ~45% vs base without going over-silent