Qwen3.8-27B-heretic-dflash2 ⚡
Qwen3.8-27B-heretic-dflash2 is a second-generation DFlash 2 speculative drafter tailored and fine-tuned for Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized Apple Silicon MLX variants like MXFP4).
Built upon the state-of-the-art DFlash 2 speculative decoding architecture, this drafter leverages Grouped Dynamic Causal Convolutions (GroupedDynamicCausalConv), Multi-Layer Hidden Representations ($5 \times 5,120 = 25,600$ dims), and Candidate Selector Path Scoring to provide ultra-fast non-autoregressive drafting with exact target model output fidelity.
🚀 Live Speculative Benchmarks (Apple Silicon MLX on ml-2)
Evaluated with mlx-vlm serving Qwen3.8-27B-heretic-ara-mxfp4 under standard temperature ($T=0.7$, max_tokens=384):
| Task / Domain | Prompt Scenario | Speculative Acceptance ($k$) | Acceptance Rate (%) | Decode Speed | Generation Time |
|---|---|---|---|---|---|
| Async Python Coding | Concurrent rate-limited web scraper with retry backoff & error handling | 2.66 tok / step | 108.8% | 13.49 tok/s | 28.46s |
| Math & Reasoning | Recursive sequence formula derivation & proof ($\sum_{n=1}^{100} \frac{1}{a_n}$) | 3.34 tok / step | 106.1% | 13.59 tok/s | 28.26s |
| Distributed Systems | Key-value store architecture with LSM compaction & Raft consensus | 2.32 tok / step | 107.9% | 11.88 tok/s | 32.31s |
- Average Speculative Throughput: ~13.0–13.6 tokens/sec on 27B model weights running on unified memory.
- First-Token Acceptance Accuracy: 75.9% on held-out validation benchmark sequences.
- Block Accuracy ($k=7$): 33.7% full 7-token exact block matching.
📊 Comparison: Base DFlash2 vs. Heretic DFlash2 (Round 1)
| Benchmark Prompt | Base z-lab/Qwen3.8-27B-DFlash2 | Trained jfan/Qwen3.8-27B-heretic-dflash2 (Round 1) |
|---|---|---|
| Python Async Scraper | 13.46 tok/s ($k=2.92$) | 13.49 tok/s ($k=2.66$) |
| Math Recursive Proof | 14.51 tok/s ($k=3.58$) | 13.59 tok/s ($k=3.34$) |
| Distributed Architecture | 12.78 tok/s ($k=2.87$) | 11.88 tok/s ($k=2.32$) |
Note: The model is undergoing autonomous multi-round scaling (Rounds 1 → 17) to further specialize token transition probability paths on Heretic / ARA distributions.
📈 Multi-Round Continuous Scaling Progress
| Checkpoint Stage | Cumulative Sequences | Training Steps | Held-Out Val Loss | 1st Token Acc | All-7 Block Acc | Status |
|---|---|---|---|---|---|---|
Pretrained Base (z-lab) | 0 | 0 | — | ~60.0% | ~23.0% | Initial Weights |
| Round 1 (Current Checkpoint) | 3,000 | 3,000 | 2.6288 | 75.9% | 33.7% | Live on HuggingFace |
| Round 2 – 4 | 12,000 | 12,000 | Training | Training | Training | In Progress |
| Round 5 – 17 | 51,000 | 51,000 | Queued | Queued | Queued | Continuous Pipeline |
🛠️ Architecture & Innovations in DFlash 2
- Grouped Dynamic Causal Convolutions (
GroupedDynamicCausalConv): Two-stage causal convolutions wrapped around self-attention and feed-forward blocks to capture local phrase structures before and after attention mixing. - Candidate Selector Path Scoring:
Dual codebooks (
predecessor_codebookandsuccessor_codebook) predicting path likelihoods across graph branches. - Multi-Layer Target Fusion:
Concatenated hidden states from Qwen 3.8 27B layers
[5, 19, 33, 47, 61]($25,600$ feature dimensions). - Optimized 8-Token Draft Block: 7 parallel draft tokens per block ($N=8$) minimizing tail verification penalty while maximizing end-to-end token generation rates.
💻 Deployment with MLX-VLM on Apple Silicon
Launch mlx_vlm.server with --draft-model and --draft-kind dflash:
python3 -m mlx_vlm.server \
--host 0.0.0.0 \
--port 8080 \
--model /path/to/Qwen3.8-27B-heretic-ara-mxfp4 \
--trust-remote-code \
--kv-bits 4 \
--kv-quant-scheme uniform \
--kv-group-size 64 \
--draft-model jfan/Qwen3.8-27B-heretic-dflash2 \
--draft-kind dflash
Once running, standard OpenAI-compatible /v1/chat/completions API calls will automatically utilize DFlash 2 speculative drafting!