Qwen3.5-4B PerfectBlend DSpark

This is a five-layer DSpark speculative decoding draft model for Qwen/Qwen3.5-4B. It is the best checkpoint from the original five-epoch training run (epoch 5, zero-based checkpoint directory 4).

The model proposes eight speculative tokens per verification step and consumes five verifier hidden states from layers 19, 23, 27, 29, and 31.

Architecture

Autoregressive acceptance results

The primary evaluation used real autoregressive vLLM decoding. Accepted tokens were fed back into the subsequent decoding context; these are not teacher-forced measurements.

Evaluation configuration:

• Dataset: RedHatAI/speculator_benchmarks
• 9 standard subsets, 80 requests per subset (720 total)
• Temperature: 0.0
• Maximum concurrency: 8
• Maximum completion length: 4,096
• Maximum model context: 32,768
• Speculative tokens: 8
• Serving vLLM: 0.28.1rc1.dev267+g46c8a161f

Acceptance length is 1 + accepted_draft_tokens / draft_steps.

Aggregate positional acceptance

The following rates are weighted by the number of draft steps in each subset.

Across all proposed draft tokens, 39.26% were accepted.

Probabilistic-sampling stress test

The same checkpoint was also evaluated with the request temperature omitted, which makes vLLM use temperature 1.0, top-p 1.0, and top-k 0.

This model was trained entirely on temperature-0 trajectories, so the greedy result is the matched evaluation. Users whose deployment relies on probabilistic sampling should benchmark the exact production temperature, top-p, and top-k.

Training data

The processed training dataset contains 10,000 examples from shanjiaz/qwen3_5_4b_perfectblend_regen. All trajectories were generated at temperature 0.0. Reasoning effort was approximately balanced:

Of the 10,000 source responses, 7,142 ended with finish_reason=stop and 2,858 ended at the generation-length limit.

Training validation

The saved epoch-5 validation metrics were:

The validation acceptance length is an in-domain, teacher-forced analytical estimate based on target/draft distribution overlap. It should not be compared directly with live autoregressive acceptance length.

Three additional low-learning-rate epochs were tested separately. That continuation lost to this checkpoint on seven of the nine matched autoregressive workloads, so this epoch-5 checkpoint is the recommended version.

Serving with vLLM

This model requires a vLLM build with DSpark support. The evaluation used:

vLLM 0.28.1rc1.dev267+g46c8a161f

Example command:

CUDA_VISIBLE_DEVICES=0 \
python -m vllm.entrypoints.cli.main serve Qwen/Qwen3.5-4B \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --speculative-config '{"method":"dspark","model":"shanjiaz/qwen3_5_4b_perfectblend_regen_dspark","num_speculative_tokens":8}'

For the matched greedy configuration, send "temperature": 0 in generation requests. Speculative decoding preserves the target model's output distribution; the draft model changes performance, not target-model semantics.

Reproducibility

• Training speculators commit: ddf9d6012b169849ea1dc53faffa41968b4e8f85
• Evaluation speculators commit: c59c5fbdf0a50a778188be43dcda54b40fff7ca2
• Training seed: 42
• Training epochs: 5
• Learning rate: 3e-4
• Packed sequence length: 8,192