Qwen3.6-Qworus-35B-A3B-APEX-GGUF
What is this?
This is a DARE-TIES merge of the following finetuned & post-trained models:
EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF: The base model, for terminal use, coding, software architecture, planning. Also provides MTP support.Kwaipilot/KAT-Coder-V2.5-Dev: Contributes efficient coding with fewer tokens over more languages.Accio-Lab/occamy-1.0: Contributes general agent abilities, also a semi-official finetune.AllSpark-Research/CompoWorld: Also contributes general agent abilities, different strengths/weaknesses from occamy.
The result is an efficient hybrid model for coding, terminal agent work, planning, designing, and conversation.
Versions:
V1.1: Revised recipe which brings in the CompoWorld 35B-A3B finetune and gives more weight to KAT-Coder and less bias towards Ornith; this version is significantly better at coding and has a wider spread of abilities.
V1: Initial version which is good at terminal use, but not as proficient in coding tasks.
Quantization
Weights are provided quantized with APEX, which preserves sensitive MoE tensors. I-Quality is the best size for high quality coding and reasoning, but I may add other sizes later. The importance matrix was calibrated on about 50% random English text snippets from multiple domains, and 50% tokens generated during real long-context agent sessions with KAT-Coder and Ornith-1.5.
Merge Recipes
Most of the model's weights were merged with DARE-TIES, except for the router gates, shared experts, and self attention layers which are protected from sparsification by using a density of 1.0 and epsilon of 0.0. A mergekit-style YAML was used to produce this model. However, the actual merge was performed by Alloy, rather than Mergekit, as the latter kept producing a broken model which could not be quantized.
V1.1 Recipe:
merge_method: dare_ties
base_model: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
tokenizer_source: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
dtype: bfloat16
out_dtype: bfloat16
models:
- model: Kwaipilot/KAT-Coder-V2.5-Dev
parameters:
weight: 0.60
- model: Accio-Lab/occamy-1.0
parameters:
weight: 0.20
- model: AllSpark-Research/CompoWorld
parameters:
weight: 0.20
parameters:
int8_mask: true
normalize: true
lambda: 1.05
density:
- value: 0.6
- filter: ".*(mlp\\.gate|mlp\\.shared_expert|self_attn\\.[qkvo]_proj).*"
value: 1.0
epsilon:
- value: 0.12
- filter: ".*(mlp\\.gate|mlp\\.shared_expert|self_attn\\.[qkvo]_proj).*"
value: 0.0
V1 Recipe:
merge_method: dare_ties
base_model: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
tokenizer_source: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
dtype: bfloat16
out_dtype: bfloat16
models:
- model: Kwaipilot/KAT-Coder-V2.5-Dev
parameters:
weight: 0.50
- model: Accio-Lab/occamy-1.0
parameters:
weight: 0.30
- model: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
parameters:
weight: 0.20
parameters:
int8_mask: true
normalize: true
lambda: 1.05
density:
- value: 0.7
- filter: ".*(mlp\\.gate|mlp\\.shared_expert|self_attn\\.[qkvo]_proj).*"
value: 1.0
epsilon:
- value: 0.12
- filter: ".*(mlp\\.gate|mlp\\.shared_expert|self_attn\\.[qkvo]_proj).*"
value: 0.0