Qwen3.6-KAT-BigBang-SLERP

English — 中文 | 父模型英文模型卡见 BigBang-README.md 与 README.source-KAT-Coder.md

Merge Details

• Method: Tensor-wise flattened SLERP at t=0.5 (consistent with mergekit's SLERP implementation), computed in float32 then converted to BF16 via round-to-nearest-even. Near-collinear tensors (|cos| > 0.9995) fall back to linear interpolation for numerical stability.
• Architecture: Qwen3.6-35B-A3B MoE — 40 layers of hybrid linear attention / full attention, 256 routed experts per layer, 262K default context.
• MoE Layout Conversion: The two checkpoints serialize expert weights differently. KAT uses per-expert tensors (experts.E.{gate,up,down}_proj.weight), while BigBang uses stacked tensors (experts.gate_up_proj [256,1024,2048], experts.down_proj [256,2048,512]). During merging, BigBang's stacked tensors are per-slice unpacked; the gate/up row ordering was empirically validated via remote small-cipher cosine probing (correct split cos ≈ 0.997, misaligned ≈ 0.00, see merge_manifest.json).
• Tensor Coverage: 613 non-expert tensors were merged by exact name/shape matching; all 30,720 expert tensors were generated from stacked tensor slicing. The visual tower unique to BigBang (333 tensors) and MTP head (19 tensors) were excluded due to the absence of corresponding KAT tensors — this model is text-only, with no unilateral modules forcibly grafted onto the result.
• Disk-friendly streaming pipeline: BigBang shards were streamed via a bounded download window (≤ 4 files on disk at a time), with SHA-256 verification against the ModelScope manifest for each; every output shard was written, fsynced, hashed, and structurally validated before deleting both fully-consumed shards. The entire process includes resume logging.
• Both original inputs were opened read-only; outputs were structurally validated and had SHA-256 computed per shard (see merge_manifest.json).

Files

If LM Studio fails to recognize the model architecture, you can manually rename to Qwen3.6-35B-A3B-KAT-BigBang-SLERP*.gguf

Evaluation

APEX Nano Real-world Test (Memory-constrained Environment)

To verify usability under memory-constrained environments, real-world testing was conducted on the full suite of 561 scenarios in ZxBench using the APEX Nano GGUF (3 attempts per question):

• Average Score: 68.86; Passed: 402/561
• Average Generation Speed: 66 tokens/s
• Test Environment: Tesla V100-SXM2 16 GB, llama.cpp PR27140 prefill fix build, 64K context, q4_0 KV cache
• VRAM Strategy: Full GPU inference, no CPU offload; Nano was chosen as the quantization version to fit that VRAM limit

Detailed scores below (average score is ZxBench's dimension-weighted average; passing threshold is total score ≥ 60):

The above results represent only the APEX Nano performance under this memory-constrained configuration and should not be taken as the upper bound of the merged model's capabilities. On devices with more VRAM, larger APEX quantization versions may yield better quality/task scores, but actual gains depend on quantization specs, context length, and hardware — please benchmark yourself.

Official scores from parent models for reference:

• KAT-Coder-V2.5-Dev: Agentic Coding SOTA among same-scale models, SWE-bench Verified 69.40 (compared to the base model Qwen3.6-35B-A3B at 64.40).
• BigBang-V1: Achieved top scores among 35B-class models on 8 benchmarks including BrowseComp, SWE-Bench Pro, FrontierScience, HLE, and MLE-Bench; overall performance between DeepSeek V4 Flash (284B) and DeepSeek V4 Pro (1.6T).

Usage (llama.cpp)

llama-server -m Qwen3.6-KAT-BigBang-SLERP-Q4_K_M.gguf \
  --alias kb -c 131072 -ngl 99 --jinja \
  --host 127.0.0.1 --port 8080

Requires llama.cpp b10516 or later (qwen3_5_moe / QWEN35MOE architecture support for hybrid linear attention layers). When started with --jinja, the thinking process is returned separately in the reasoning_content field in the OpenAI-compatible API.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="kb",
    messages=[{"role": "user", "content": "Implement an async cache with LRU eviction in Python, and explain the design trade-offs."}],
    temperature=0.6, top_p=0.95,
)
print("Thinking:", getattr(response.choices[0].message, "reasoning_content", None))
print("Answer:", response.choices[0].message.content)

Recommended sampling parameters: temperature=0.6, top_p=0.95, top_k=20. Since BigBang training made heavy use of long context, complex tasks should maintain ≥ 128K context to fully preserve thinking capability.

Sources and License

• Common ancestor: Qwen/Qwen3.6-35B-A3B — both parent models are post-trained variants of this base model (merged with equal weight, so no single parent is designated as the base model).
• Structure source: Kwaipilot/KAT-Coder-V2.5-Dev (Apache-2.0; open-weight version contains only language model weights, text-only)
• Other source: endless-frontier/BigBang-v1 (Apache-2.0)
• Original model weights: Dalvlad/Qwen3.6-KAT-BigBang-SLERP-35B
• Released under Apache-2.0; the corresponding licenses of both source checkpoints have been respected. Model merging is an experimental technique and does not guarantee preserving either parent model's benchmark scores or safety performance — please evaluate before use.

Acknowledgments

• Inference powered by llama.cpp; merging uses a lightweight NumPy SLERP implementation equivalent to mergekit, augmented with custom stacked-MoE unpacking and a disk-friendly streaming merge pipeline.
• Thanks to ZxBench for the deterministic evaluation pipeline and 561-scenario test set.
• Thanks to Kwaipilot and endless-frontier for open-sourcing two excellent parent models.

Limitations

• This model is a text-only export: it does not include the visual tower (BigBang's multimodal components were not incorporated) nor the MTP speculative decoding head.
• Q4_K_M is lossy quantization; for tasks sensitive to precision, BF16 weights are recommended.
• Has common limitations of 35B-A3B MoE models: hallucination, unstable long-range planning, sensitivity to prompt formatting.
• The merge has not been systematically evaluated, and the degree of capability fusion from both parent models is unknown.

Citation

@misc{qwen36_kat_bigbang_slerp,
  title  = {Qwen3.6-KAT-BigBang-SLERP},
  author = {Local merge, tensor-wise flattened SLERP t=0.5 with stacked-MoE unpacking},
  year   = {2026},
  note   = {Merged from Kwaipilot/KAT-Coder-V2.5-Dev and endless-frontier/BigBang-v1}
}