Qwen3.6-35B-A3B — TurboQuant+ Config-I (GGUF, experimental)

Experimental Config-I quantization of Qwen/Qwen3.6-35B-A3B for llama.cpp, produced with the feature/turboquant-kv-cache branch.

Status: experimental. Metal coherency looks clean across contexts (see samples below). CUDA has not been tested yet — if you run this on a CUDA build of the fork, please open an issue with your results. Use at your own risk.

Details

Runtime

This GGUF is intended for TheTom's llama.cpp fork. Upstream llama.cpp does not (yet) support the TQ3_1S weight type or the TurboQuant asymmetric KV path, so loading/running will likely fail outside the fork.

git clone https://github.com/TheTom/llama.cpp
cd llama.cpp
git checkout feature/turboquant-kv-cache
cmake -B build -DGGML_METAL=ON && cmake --build build -j
build/bin/llama-bench -m Qwen3.6-35B-A3B-ConfigI.gguf -p 128 -n 32 -ngl 99

Coherency (Metal, feature/turboquant-kv-cache)

Sanity generations on an M5 Max 128GB using the production KV recipe (-ctk q8_0 -ctv turbo3 -fa 1). Prompt is the opening line of The Great Gatsby with "Summarize the passage:" appended. Reasoning traces preserved; responses trimmed for brevity. The model correctly identifies Gatsby at every context length and flags the missing passage body.

ctx ≈ 128

<think>
1. Analyze User Input:
   - Quote: "In my younger and more vulnerable years my father gave me some advice."
   - Request: "Summarize the passage:"
   - Key Observation: This is the famous opening line of F. Scott Fitzgerald's
     The Great Gatsby. The user is asking to "summarize the passage," but only
     provided a single sentence.
</think>

ctx ≈ 1024

The phrase you provided is the opening line of F. Scott Fitzgerald's novel
*The Great Gatsby*.

Since you only provided the opening sentence, here is a summary of its
significance and context:

Summary of the Passage (Opening Line)
> "In my younger and more vulnerable years my father gave me some advice."

This sentence establishes the narrator's reflective tone …

ctx ≈ 4096

<think>
1. Analyze User Input:
   - The user quotes the famous opening line: "In my younger and more
     vulnerable years my father gave me some advice."
   - They ask: "Summarize the passage:"
   - Wait, there's no passage provided after the colon. The user only
     provided the opening line and then asked for a summary of "the passage,"
     but didn't include the passage body.
</think>

ctx ≈ 16384

<think>
1. Analyze User Input:
   - The user provides a famous opening line: "In my younger and more
     vulnerable years my father gave me some advice."
   - They ask to "Summarize the passage:"
   - But there's no passage provided! The prompt is incomplete.
2. Identify Key Issue:
   - The user quotes the opening line of F. Scott Fitzgerald's The Great …
</think>

ctx ≈ 32768

The passage "In my younger and more vulnerable years my father gave me some
advice" is the famous opening line of F. Scott Fitzgerald's novel
*The Great Gatsby*.

It sets the tone for the entire book, which is a first-person retrospective
narrative. The narrator, Nick Carraway, looks back on his experiences in
Long Island and New York during the summer of …

Metal verdict. Coherent and on-topic at every context length tested, including the 32k full-context run. No drift, no repetition, no nonsense. Quality looks good at this compression level based on coherence alone — a full PPL/MMLU sweep has not been run.

CUDA: not tested yet. If you run this on a CUDA build of the fork, please open an issue on the fork with your results.

Links

• Config-I paper
• TurboQuant+ repository
• MLX counterpart (same policy, standard MLX format): thetom-ai/Qwen3.6-35B-A3B-ConfigI-MLX

Quantized by @thetom-ai · GitHub · X · Sponsor