Qwen3.8-Whittle-w50 (18.3B, unrepaired)
☕ Support this work
Whittle is built by one person on a grocery budget and rented GPU hours. If this research is useful to you, or you want to see it finished: ko-fi.com/davida81328. Every hour of GPU time goes straight into the next checkpoint, and every checkpoint, table and log lands in these repos.
Uniform 50% MLP width prune of Qwen3.8-27B. Neuron selection per layer by measured strength (down-projection column norm times activation standard deviation over a mixed corpus), all 64 layers pruned including the entry layer and the deep tail. All 16 full-attention stations, the full stop-signal stack, and every knowledge block remain in place at half width.
Status
Earlier line (Aug 2026); the current Whittle models are the Whittle-Next / Whittle-Qwen-3.8 line: Whittle-Qwen-3.8-35B-A3B.
Research preview: the other pole of the compression design space — all 64 layers kept, every MLP halved. No repair training yet. Everything is still in there, just dull. A one hour sharpening run is prepared (see Training pack). It needs post training and is published as the record of a compression method and its measurements rather than as a finished assistant. Further healing and instruction tuning runs are planned over time, and these checkpoints will improve as those land.
Which weights are which
| path | what it is |
|---|---|
model-*.safetensors, config.json, tokenizer (repo root) | the pruned, unrepaired weights in bf16 (the only weights in this repo; no GGUF is published here) |
training/ | the prepared sharpening run: train_heal_q38.py, heal60_mix.jsonl, colab_w50_sharpen_cell.txt |
Measured
The depth-cut branch of the Whittle family (44L and 48L cuts) removes whole layers: fast, sharp, but the removed knowledge is measurably GONE. This variant tests the opposite bet: keep every layer, thin every layer. The knowledge atlas and a forced-choice probe say the bet pays:
| measurement | depth-restored 18.3B | width-w50 18.3B (this) |
|---|---|---|
| 39-prompt recall battery | 34/39 | 27/39 |
| loop generations (DRY off) | 0/6 | 1/6 |
| forced-choice recognition | 5/7 (facts gone) | 6/7 (facts dormant) |
Same parameter count, opposite failure styles: the depth cut is sharp on what it kept and blank on what it dropped; this model answers everything a little worse. Recognition beats recall by a wide margin here, which means the knowledge survives at half width and is waiting for a short sharpening run. A sibling variant that spared blocks 8-11 at full width scored 7/7 on recognition, confirming the pattern.
Training pack (in this repo)
training/ holds everything for the prepared one hour A100 sharpening run:
train_heal_q38.py (QLoRA nf4, r64 all-linear including the GDN projections),
heal60_mix.jsonl (~10M tokens: 4096-token complete book pages, fenced code
drills, multi-turn tool-format sessions, fact and arithmetic anchors), and the
Colab cell in colab_w50_sharpen_cell.txt. Sequence length 8192, wall-clock
capped. All mix sources are clean lineage (public domain, MIT, or ours).
Run it
Runs in the Whittle fork of llama.cpp (per-layer FFN width support; uniform
width also loads in stock llama.cpp). Anti-loop sampling recommended:
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 4 --repeat-penalty 1.15
Caveats
Known weaknesses in the unrepaired state: dulled confidence on hard recall (the intact model answers "Paris" at ~0.5 probability, this one at 0.026, top-1 still correct), occasional empty outputs on code prompts, markdown fencing unreliable.
Support this work
Independent research on consumer hardware. Every donation becomes A100 hours, and every A100 hour ends up as a public model or a public measurement. ☕ ko-fi.com/davida81328
Provenance, licence and authors
Base model by the Qwen team (Apache 2.0). Pruned and measured by David Aylward with Claude (Fable 5, Anthropic) as co-author.