Qwen3.6-35B-A3B-NSC-ACE-SABER GGUF
This repository hosts llama.cpp/GGUF builds for
GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER. The source checkpoint is the full
safetensors model in GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER.
The quantization suite is being built down to Q2_K. Higher quants preserve more of the SABER-selected distribution; lower quants are provided for smaller VRAM/RAM targets and should be evaluated against your own prompts.
Image/video sidecars: This repository now includes the restored Qwen3.6 multimodal config, processor/preprocessor files, tokenizer/chat template, safetensors index, and
model-vision-from-qwen3.6-base.safetensorsvisual tower sidecar. The existing.ggufbinaries were not rewritten in this metadata-copy pass.
Release Snapshot
| Item | Value |
|---|---|
| Source checkpoint | GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER |
| Base model | Qwen/Qwen3.6-35B-A3B |
| Format | GGUF for llama.cpp-compatible runtimes |
| Quantization range | F16, Q8_0, Q6_K, Q5_K_M, Q5_K_S, Q4_K_M, Q4_K_S, Q3_K_L, Q3_K_M, Q3_K_S, Q2_K |
| Final source compliance | 98.33% on HarmBench-300 |
| Final source KLD | 0.025383937664711 |
Benchmark Plots
BFCL Tool-Calling Check
The source safetensors checkpoint was compared against Qwen/Qwen3.6-35B-A3B
on a 40-case BFCL subset: 20 simple and 20 multiple-function prompts. GGUF files
inherit from that checkpoint, but individual quants should be rechecked if exact
tool-call behavior matters.
| Metric | Base | NSC-ACE SABER source |
|---|---|---|
| Tool-call rate | 92.50% | 95.00% |
| Function name accuracy | 92.50% | 95.00% |
| Required argument name accuracy | 90.00% | 93.12% |
| Required argument value accuracy | 79.79% | 83.54% |
| Exact required-call accuracy | 75.00% | 77.50% |
Available Files
| File | Status | Notes |
|---|---|---|
Qwen3.6-35B-A3B-NSC-ACE-SABER-F16.gguf | uploaded | Full GGUF conversion source / highest local fidelity |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q8_0.gguf | uploaded | Near-full quality, large local file |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q6_K.gguf | uploaded | High-quality local default if memory allows |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q5_K_M.gguf | uploaded | Strong quality/size balance |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q5_K_S.gguf | uploaded | Smaller Q5 option |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q4_K_M.gguf | uploaded | Common balanced local target |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q4_K_S.gguf | uploaded | Smaller Q4 option |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q3_K_L.gguf | uploaded | Lower-memory Q3 option |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q3_K_M.gguf | uploaded | Smaller Q3 balance |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q3_K_S.gguf | uploaded | Small Q3 option |
Qwen3.6-35B-A3B-NSC-ACE-SABER-Q2_K.gguf | uploaded | Minimum-size target; quality loss expected |
The uploader refreshes this card as each artifact finishes. Uploaded non-F16 files are deleted from the build pod after upload to stay under the pod volume quota.
Which Quant Should I Use?
| Quant | Best fit |
|---|---|
| F16 | Maximum fidelity when disk/RAM are not a concern |
| Q8_0 | Very high fidelity local inference |
| Q6_K | Recommended high-quality local starting point |
| Q5_K_M | Strong balance for quality and size |
| Q4_K_M | Practical default for constrained machines |
| Q3_K_M / Q3_K_S | Low-memory experiments |
| Q2_K | Smallest target; use only when memory is the hard constraint |
For agentic/tool-calling workloads, prefer Q6_K, Q5_K_M, or Q4_K_M when possible. Very low quants can affect formatting, argument fidelity, and refusal calibration.
What NSC-ACE Is
NSC-ACE means Neural Steering Committee for Agentic Co-Evolution. The source checkpoint was trained by generating multiple steered rollouts from the same model and rewarding convergence in tool-call structure across those latent modes. The goal is stronger agentic/tool-use behavior: stable function selection, argument filling, useful reasoning wrappers, and fewer repeated tool loops.
SABER was applied after NSC-ACE as a calibration stage. The release objective was to raise HarmBench-300 compliance while keeping KLD and PPL movement low.
Running With llama.cpp
llama-cli \
-m Qwen3.6-35B-A3B-NSC-ACE-SABER-Q5_K_M.gguf \
-c 32768 \
-ngl 999 \
-p "Write a compact tool plan for indexing a Python repo."
For OpenAI-compatible local serving:
llama-server \
-m Qwen3.6-35B-A3B-NSC-ACE-SABER-Q5_K_M.gguf \
-c 32768 \
-ngl 999 \
--jinja
Use a current llama.cpp build. Qwen3.6 support, chat-template handling, and tool-call behavior depend on runtime freshness.
Quantization Notes
- GGUFs are produced from the accepted full safetensors checkpoint.
- The source model's final release metrics are measured before quantization.
- Quantized files should be re-evaluated if exact compliance/KLD behavior matters.
- Lower bitrate files can degrade structured output before they obviously degrade prose quality.
Related Repositories
- Full safetensors checkpoint:
GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER - Base model:
Qwen/Qwen3.6-35B-A3B