hauhau_qwen36_35b_a3b_mxfp4_gguf

This repo contains a local MXFP4_MOE GGUF conversion of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive, prepared with a patched local llama.cpp toolchain.

Files

• hauhau_qwen36_35b_a3b_mxfp4_gguf.gguf: text-only GGUF export in robinshao-style MXFP4 layout
• llama.cpp.src-patched.zip: patched local llama.cpp source snapshot used for conversion and serving
• build_result.json: conversion and quantization summary
• api-benchmark.json: basic local API benchmark output

Quantization Notes

• Source FP16 weights: khoi11/Qwen3.6-35B-A3B-Uncensored-HauhauCS-FP16
• Upstream model lineage: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
• Converted to GGUF with local convert_hf_to_gguf.py --outtype f16 --no-mtp
• Vision tensors are not included in the final GGUF
• MTP tensors are not included in the final GGUF
• Quantized with local llama-quantize using MXFP4_MOE --pure --output-tensor-type q8_0 --token-embedding-type q8_0

Tensor Layout

{
  "tensor_count": 733,
  "type_counts": {
    "Q8_0": 2,
    "F32": 301,
    "MXFP4": 430
  },
  "output_bytes": 19041835392
}

Local Deployment Notes

This file was validated locally with patched llama.cpp on Windows using:

llama-server -m hauhau_qwen36_35b_a3b_mxfp4_gguf.gguf \
  --jinja \
  --device CUDA0 \
  --gpu-layers all \
  --ctx-size 110592 \
  --batch-size 768 \
  --ubatch-size 128 \
  --flash-attn on

Local API benchmark on an RTX 4000 Ada at ctx-size=110592 observed roughly 103-105 tok/s decode speed for short requests.

Caveats

• This is a text-only export, not the original multimodal package.
• Native training context metadata remains 262144, but the local validation run used 110592.
• Behavior quality and multilingual stability may vary by prompt and runtime configuration.