Qwen3.8-27B-Opus-Distill-v2
Qwen/Qwen3.8-27B fine-tuned (LoRA, merged) on verified-genuine Claude Opus
reasoning traces. This release fixes a reasoning-loop bug reported against
the first release (barozp/Qwen3.8-27B-Opus-Distill,
"v1") while keeping the reasoning-quality gains v1 was known for.
Why this release exists
A community member (zxbc2023) reported a
clean, deterministic bug on v1: certain stacked output-format constraints
(most notably "no prose" combined with a second constraint like "no markdown") sent the model into a non-converging self-verification loop —
burning the entire token budget on repeated "wait, let me re-check…" passes
with zero visible output. Their repro was exact and reproducible at
temp=0, and they correctly diagnosed it as likely inherited from the
distillation data rather than a capability gap. Full credit to them for the
report that made this fix possible.
Root cause. Investigating v1's training data (opus-reasoning-distill-train)
showed that 83% of it had actually been sourced from Jackrong's
TraceInversion datasets — reasoning traces reconstructed by a small
auxiliary model (Trace-Inverter-4B) from a compressed summary, not the
teacher model's real output (openly documented on those datasets' own
cards). That reconstruction process appears to have baked in a rigid
"enumerate constraints, then re-verify" template — plausibly both the
source of v1's reasoning-quality gains and the loop bug, two sides of the
same mechanism.
The fix, in two steps:
- First attempt: retrain purely on a genuine-trace dataset (
lordx64/reasoning-distill-claude-opus-4-7-max). This fixed the loop bug completely but collapsed the reasoning-transfer gain (GPQA-Diamond dropped back to base/random level) — the domain mix mattered as much as trace authenticity. - This release: reconstructed v1's exact prompt distribution, but
replaced every row traced back to Jackrong's fabricated sets with the
verified genuine trace for the same prompt (from
lordx64orRoman1111111/claude-opus-4.6-10000x). Then ran a full-dataset scan (30 parallel LLM-judge passes over all 14,250 rows) for the specific pathological pattern and removed every flagged row (169) plus every row whose source was never verified against a known-genuine dataset (2,429). Net: 14,250 → 11,716 rows, 100% traced to verified sources.
Bug fix validation
Re-ran zxbc2023's exact repro (temp=0, seed=1234, fib(n) with stacked
constraints) against this model:
| Test | v1 | v2 |
|---|---|---|
| "no prose" + "no markdown" | 3000/3000 tokens, 0 visible output | 87/4096 tokens, clean code |
| "no prose" + "no comments" | 3000/3000 tokens, 0 visible output | 73/4096 tokens, clean code |
| Generalization (different task, same constraints) | — | 43/4096 tokens, clean code |
All three converge quickly with </think> closing normally — no loop, no
budget exhaustion.
Benchmark results
Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice),
chat template OFF, QUICK mode (--limit 500) — identical protocol to v1,
so the Δ column is the meaningful signal.
| Task | Metric | Base | v2 | Δ | (v1's Δ, for reference) |
|---|---|---|---|---|---|
| wikitext | word perplexity ↓ | 8.4335 | 8.3788 | −0.055 | −0.09 |
| mmlu | acc | 0.8494 | 0.8476 | −0.002 | −0.001 |
| hellaswag | acc_norm | 0.7420 | 0.7500 | +0.008 | −0.002 |
| arc_challenge | acc_norm | 0.5880 | 0.6220 | +0.034 | +0.042 |
| gpqa_diamond | acc_norm | 0.2323 | 0.4697 | +0.237 | +0.263 |
Reading the table: knowledge (MMLU) and language modeling (wikitext) are unaffected; reasoning (ARC, GPQA) shows a large, real gain — slightly smaller than v1's (expected: v1's gain was partly driven by the same fabricated content that caused the bug), but still far above noise. GPQA in particular is a same-protocol delta vs. base, not comparable to Qwen's own published 89.2 (thinking-mode-on, different harness) — see v1's card for the full caveat.
Model overview
- Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
- Method: LoRA (
r=64, alpha=64, dropout=0.05), merged into the base weights. - LoRA targets: attention
q/k/v/o_projon the 16 full-attention layers only; FFNgate/up/down_projon all 64 layers. - Training data:
barozp/opus-reasoning-distill-v2(11,716 examples, 100% verified-genuine Opus traces). - Training run: 1 epoch (696 steps),
lr=1e-4cosine with 3% warmup, effective batch 16,MAX_SEQ=4096, bf16, ~4h15m on an A100 80GB. - Final validation loss:
0.4334— lower than both v1 (0.4647) and the intermediate pure-genuine attempt (0.805), despite less data. - Vision + MTP: carried over from base, never trained.
Quick start
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch
model = AutoModelForImageTextToText.from_pretrained(
"barozp/Qwen3.8-27B-Opus-Distill-v2", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill-v2")
Text-only inference also works via AutoModelForCausalLM.
Related
- Qwen3.8-27B — the base model.
- barozp/Qwen3.8-27B-Opus-Distill — v1. Still works well for most use cases; this release specifically fixes the stacked-constraint reasoning-loop edge case.
- barozp/opus-reasoning-distill-v2 — the training dataset, with full provenance notes.
- Thanks to zxbc2023 for the bug report that led to this release.
Quantized releases of this checkpoint:
- barozp/Qwen3.8-27B-Opus-Distill-v2-FP8 -- block-wise FP8 (e4m3, 128x128) for CUDA serving stacks; paired BF16-vs-FP8 evals show deltas within noise.
- barozp/Qwen3.8-27B-Opus-Distill-v2-GGUF -- llama.cpp / Ollama / LM Studio quants (BF16 down to IQ1_M), mmproj vision, MTP self-speculative decoding.
- barozp/Qwen3.8-27B-Opus-Distill-v2-MLX-4bit and MLX-8bit -- Apple Silicon mlx-vlm conversions.