Qwen3.8-27B-heretic-dflash ⚡
Qwen3.8-27B-heretic-dflash is a high-performance DFlash speculative drafter trained on multi-layer hidden representations from Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized variants like MXFP4).
By predicting future token blocks in parallel using single forward-pass target hidden representations, this drafter accelerates autoregressive generation on Apple Silicon (MLX) and NVIDIA GPUs by 2.6× to 2.8× with exact target model output fidelity.
🚀 Live Speculative Benchmarks (Apple Silicon MLX)
Evaluated against Qwen3.8-27B-heretic-ara-mxfp4 on real-world prompts with standard temperature:
| Task / Domain | Prompt Scenario | Speculative Acceptance ($k$) | Acceptance Rate (%) | Decode Speed | End-to-End Speedup |
|---|---|---|---|---|---|
| Async Python Coding | Concurrent rate-limited web scraper with retry backoff & error handling | 3.01 tokens / round | 82.05% | 21.9 tok/s | 2.58× faster (vs ~8.5 tok/s baseline) |
| Math & Reasoning | Recursive sequence formula derivation & proof ($\sum_{n=1}^{100} rac{1}{a_n}$) | 4.34 tokens / round | 80.50% | 24.0 tok/s | 2.82× faster (vs ~8.5 tok/s baseline) |
- Draft Acceptance Rate: Over 81% of all proposed draft tokens are verified and accepted.
- First-Token Acceptance Rate ($k \ge 1$): >95%, eliminating stalling during token generation.
📈 Multi-Round Continuous Training Progress
The model is iteratively trained on multi-domain instruction sequences (coding feedback, mathematical reasoning, system architecture, and multi-turn chat) using pure bfloat16 extracted hidden states:
| Checkpoint Stage | Cumulative Sequences | Cumulative Tokens | Held-Out Val Loss | Block Accuracy ($k=15$) | Status |
|---|---|---|---|---|---|
| Untrained Base | 0 | 0 | 7.3247 | 7.7% | Initial z-lab weights |
| Round 1 – 2 | 6,000 | 12.4M | 3.4830 | 19.3% | Baseline Convergence |
| Round 3 – 5 | 15,000 | 31.0M | 3.8210 | 14.8% | Cross-Domain Diversity |
| Round 6 – 8 | 24,000 | 49.5M | 3.6890 | 16.2% | Reasoning & Proofs |
| Round 9 – 10 | 30,000 | 61.9M | 3.3995 | 17.8% | Current Checkpoint (Live) |
| Round 11 – 17 | 51,000 | ~105M | Queued | Queued | Full 50k Step Schedule |
🛠️ Architecture & Specifications
- Drafter Architecture: DFlash Dual-Hidden Non-Autoregressive Drafter
- Target Model: Qwen 3.8 27B Heretic / ARA
- Hidden State Feature Dimension: $5 imes 5,120 = 25,600$ (concatenated target layers in pure
bfloat16) - Context Length: 2,048 tokens + 16 block tokens
- Block Size ($N$): 16 future tokens per speculative step
- Storage Footprint:
3.46 GBsafetensors
💻 How to Use with MLX / MLX-VLM
Launch mlx_vlm.server with --draft-model and --draft-kind dflash:
python3 -m mlx_vlm.server --host 0.0.0.0 --port 8080 --model /path/to/Qwen3.8-27B-heretic-ara-mxfp4 --trust-remote-code --kv-bits 4 --kv-quant-scheme uniform --kv-group-size 64 --draft-model jfan/Qwen3.8-27B-heretic-dflash --draft-kind dflash
Once running, send standard OpenAI-compatible /v1/chat/completions requests to enjoy accelerated 22–24+ tokens/second inference.