Whittle-Qwen-3.8-45B-A3B
☕ Support this work
Whittle is built by one person on a grocery budget and rented GPU hours. If this research is useful to you, or you want to see it finished: ko-fi.com/davida81328. Every hour of GPU time goes straight into the next checkpoint, and every checkpoint, table and log lands in these repos.
The bf16 safetensors of Whittle-Qwen-3.8-45B-A3B: the Whittle flagship
35B with the 76 experts per layer that its expert prune removed put back from
Qwen3.6-35B-A3B, untrained. 45.1 B total (35.1 B body + 10.0 B n-gram memory), ~3 B active
(8 of 256 routed experts + the shared expert), Qwen3.8-Flash-Next (qwen4_exp) format. For llama.cpp use the
GGUF repo: it is much faster locally, and every measurement below was taken on it.
Try it in the browser: Whittle 45B demo (ZeroGPU).
Status
Experimental (5 Oct 2026). An untrained splice, measured on the same battery as the 35B and its base (see the GGUF card). The restored experts were trained for Qwen3.6's own body and have not been healed into this one; arithmetic word problems are the visible cost.
Load it with transformers
Stock transformers (>= 5.18) has the architecture, but plain from_pretrained would run this checkpoint with a random n-gram table: Whittle's
table uses 8 hash heads of exactly 4,880,000 rows (the geometry its GGUFs give llama.cpp), where the stock code sizes heads as primes, and it is
stored under a different key prefix. whittle_load.py in this repo sizes the table from the checkpoint's own hash buffers, maps the keys onto the
stock layout and checks the buffers after loading. On a small random model saved in this layout it reproduces the original's logits exactly.
import importlib.util, torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer
repo = "logic65/Whittle-Qwen-3.8-45B-A3B"
spec = importlib.util.spec_from_file_location("whittle_load", hf_hub_download(repo, "whittle_load.py"))
whittle_load = importlib.util.module_from_spec(spec); spec.loader.exec_module(whittle_load)
model = whittle_load.load_whittle(repo, dtype=torch.bfloat16, device_map="auto") # the ~20 GB table stays in CPU memory
tokenizer = AutoTokenizer.from_pretrained(repo)
messages = [{"role": "user", "content": "Explain hash collisions in two sentences."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=True, return_tensors="pt",
enable_thinking=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.05)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
- Memory: about 70 GB of weights on the GPU(s) and the 20 GB table in CPU memory.
- Install
flash-linear-attentionfor the gated-DeltaNet layers; without it transformers falls back to a much slower reference path. - Sampler as the GGUF card: temperature 0.7, top_p 0.8, top_k 20, repetition penalty 1.05; do not decode greedily. Thinking:
enable_thinking=Truein the chat template, with 4k+ new tokens for code and 8k+ for maths.
Measured
On the GGUF Q4_K_M, same harness as the 35B and Qwen3.6-35B-A3B (full table and method on the GGUF card): HumanEval / HumanEval+ 75.6 / 72.6, MBPP / MBPP+ 77.8 / 65.9, tool-use probes 96/96 (long-context 24/24), MATH-60 44/60, long-tail names 111/252 (35B: 94), GSM-50 38/50 (35B: 44 - a real drop, 6 lost and none gained on the same problems).
How it was made
The same recipe as the GGUF, on the original tensors: the 35B root's safetensors (Phase-2 step 32010) with, in every layer, Qwen3.6-35B-A3B's
bf16 weights of the 76 pruned experts appended as experts 180-255 (sorted original order; keep-lists in
eval/prune_report.json) and their router rows
appended, scaled per layer by the median norm ratio of the trained to the original kept rows (1.001-1.026, median 1.009). The kept experts still
sit at cosine 0.982-0.997 to their Qwen3.6 originals (gate, up and down projections, checked at layers 0, 10, 20, 30 and 39) and the trained router
rows at 0.978-0.995, so the restored experts meet the body in the space they were trained for. num_experts 180 -> 256; nothing else changed and nothing was trained. The 35B root files also hold 80 stale duplicate copies of
router-type tensors that the index does not reference; transformers loads the indexed copies (checked).
Provenance
David Aylward (logic65) & Claude (Anthropic). Built from logic65/Whittle-Qwen-3.8-35B-A3B (teacher Qwen/Qwen3.8-27B, memory contents from Qwen/Qwen3.8-Flash-Next) and the routed experts of Qwen/Qwen3.6-35B-A3B. All Apache-2.0.