Qwen3.8-Whittle-tri (14.7B)
☕ Support this work
Whittle is built by one person on a grocery budget and rented GPU hours. If this research is useful to you, or you want to see it finished: ko-fi.com/davida81328. Every hour of GPU time goes straight into the next checkpoint, and every checkpoint, table and log lands in these repos.
A 27B compressed to 14.7B by training, not cutting. It writes code, answers factual questions, never loops, and fits on two consumer 8GB cards.
Status
Earlier line (Aug 2026); the current Whittle models are the Whittle-Next / Whittle-Qwen-3.8 line: Whittle-Qwen-3.8-35B-A3B.
Needs post training. A research preview, published as the record of a compression method and its measurements rather than as a finished assistant. Further healing and instruction tuning runs are planned over time, and these checkpoints will improve as those land. The conversational repair of this model is published separately as Qwen3.8-Whittle-tri-14.7B-chat.
Which weights are which
q38_tri_repaired_q8.gguf(15.7GB) - the model this card describes. Download it and run it; that is the whole setup.q38_tri_healed_q8.gguf- a second Q8 GGUF in the tree, not described on this card.bf16/- the bf16 safetensors (8 shards) with config, chat template and tokenizer.heal-lora/,repair-lora/- the heal and repair adapters.recipe/-assemble_relay_hf.py,pair_distill_poc.py,relay_campaign.py,train_heal_tri.py.training/-colab_repair_cell.txt,repair_mix.jsonl,repair_mix_builder.py,train_heal_tri.py.
Measured
What it is good at
Factual recall is close to intact: capitals, elements, dates, who wrote what. Code is solid across Python, C, JavaScript, SQL and HTML, and markdown fencing is reliable (8/8 on our fencing probe). It is loop-free: zero repetition failures in six generations at both greedy and temperature 0.7, which is not true of the larger models in this family.
What it is bad at
Arithmetic and multi-step reasoning. It scores 3.6% on GSM8K. It will often state the right rule and then compute the wrong answer. Do not use it for maths, data analysis, or anything where a number has to be right.
Non-English is unreliable outside French and Spanish. Long agent sessions were not the target of this model's training.
Honest positioning
39-prompt battery: 31/39. That ties a variant 2.1B larger and beats the 19.2B cut, but two models in the family score higher (36/39 at 16.8B, 35/39 at 20.8B). Pick this one if you want the smallest thing that still works, or if looping has bitten you before. Pick the bigger ones if you want maximum quality.
This is a research preview. It needs real post-training before it reasons, and that is a funding problem rather than a technical one.
How it was made, briefly
The 64 layers of Qwen3.8-27B were distilled into 32: adjacent recurrent layers were merged in pairs, and eight full-attention stations were absorbed into their neighbours rather than deleted. Nothing was dropped. Full methodology, every script, the adapters and the measurements are in the developer card: Whittle dev
Collections: models you can run and research and dev
Run it
llama-server -m q38_tri_repaired_q8.gguf -ngl 24 -c 4096 --jinja
Standard [gdn, gdn, gdn, attention] blocks at exactly 3:1, so it loads in any
llama.cpp build with Qwen3.5 support. No fork, no patches, no special flags.
On 16GB of VRAM keep about 24 of the 32 layers on GPU; with 24GB it fits whole.
Provenance, licence and authors
Independent research on consumer hardware. Every donation becomes GPU hours, and every GPU hour ends up as a public model or a public measurement. ☕ ko-fi.com/davida81328
Base model by the Qwen team (Apache 2.0). Built and measured by David Aylward with Claude (Fable 5, Anthropic) as co-author.