Qwen3.6-27B-nothink-DFlash2-v0.1
A BF16 DFlash2 draft model adapted for Qwen/Qwen3.6-27B with thinking disabled. This is a speculative-decoding companion, not a standalone chat model. Use the target model's tokenizer and a runtime implementing DFlash2DraftModel.
Checkpoint and provenance
This release is the step-100 checkpoint from a new-target adaptation run: 100 additional optimizer updates starting from the already-trained v0.3 draft, with a fresh optimizer. It was not trained from scratch. The initializer was adapted previously for a Qwen3.8/Qwen3.6 blend; this run adapts it to stock Qwen3.6-27B.
- Target:
Qwen/Qwen3.6-27B@6a9e13bd6fc8f0983b9b99948120bc37f49c13e9, BF16, thinking disabled. - Initial draft:
vvsotnikov/Qwen3.8-27B-test-DFlash2-v0.3@e6a6ef16e3a16dbee56aa3f6fee948ef64a444fa. - Initialization weights SHA-256:
fbee4cc4790035d0a4d4d7cece25f41a0f8d629b4c630d19be86a994ee90a477. - Export: 1,924,404,480 BF16 parameters in 81 tensors; 3,848,817,920 bytes.
- Export weights SHA-256:
52d968b9a9f7cc077580327074119a3815f74f48296c5f05b1aa1e0713f7b1f8.
The run was stopped before its planned 500-update budget. This release contains the saved step-100 inference export, without optimizer state or training data.
Training
Training uses recorded agentic coding trajectories generated by the target with thinking disabled. Source generation used temperature 0.7, top-p 0.8, top-k 20, and presence penalty 1.5. Token IDs and response loss masks were preserved.
| Setting | Value |
|---|---|
| Updates in this release | 100 |
| Effective batch | 4 calls: DDP 2, microbatch 1, accumulation 2 |
| Training calls processed | 400 |
| Learning rate | 1e-4; 5 warmup updates, then constant |
| Optimizer | AdamW; FP32 master weights and moments |
| Draft precision | BF16 |
| Gradient clipping | 1 |
| Objective | Recorded-token CE + coupled selector CE (alpha 1), gamma 4 |
| Anchors / objective chunks | 64 / 8 |
| Attention backend | SDPA |
| Target embeddings and output head | Frozen |
| Seed | 42 |
The selected training pool contained 2,000 calls; only 400 calls contributed to the released step-100 checkpoint. Training used SpecForge with native SGLang feature extraction and Mooncake transport. Pinned source revisions: SpecForge ed64d275bac8a48e126adb2827368603566a3029; Speculators 3419401a380db305ed6493ba4680ad8a3c3e0e45.
Architecture
Five draft layers, hidden size 5,120, intermediate size 17,408, 32 attention heads, 8 KV heads, vocabulary 248,320, block size 8, and target feature taps [5, 19, 33, 47, 61]. See config.json for the full configuration. The configured maximum position count is not a validated context-length claim.
Validation status
All 81 exported BF16 tensors were checked for exact equality with the saved training checkpoint. This step-100 checkpoint has not undergone serving benchmarks or downstream agent-quality evaluation. Results from the initializer or separate short training runs do not establish this checkpoint's acceptance rate, speed, or quality.
Attribution and license
Apache-2.0. The draft lineage originates from z-lab/Qwen3.8-27B-DFlash2, developed by Inco AI. DFlash 2 builds on DFlash by Jian Chen, Yesheng Liang, and Zhijian Liu. Qwen models were developed by Alibaba Cloud. See LICENSE, NOTICE, and CHANGES.md.