Qwen3.6-35B-A3B REAP Pruned Ratio 0.5
This repository contains a REAP-pruned version of Qwen/Qwen3.6-35B-A3B.
The checkpoint was produced with routed-expert pruning using REAP
(Router-weighted Expert Activation Pruning), which scores routed experts
with router weights and expert activation norms.
Pruning Settings
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3.6-35B-A3B |
| Compression / pruning ratio | 0.50 |
| Pruning method | reap |
| Calibration samples | 1024 |
| Calibration sequence length | 2048 |
| Seed | 42 |
| Router weight renormalization | true |
| Routed experts per MoE layer | 256 -> 128 |
| Routed experts selected per token | 8 |
| Shared experts | Preserved |
| Precision | BF16 |
| Quantization | None |
Calibration Data
The calibration set used the REAP paper/code mixture with 1024 total samples:
theblackcat102/evol-codealpaca-v1: 171 samplesSalesforce/xlam-function-calling-60k: 171 samplesopen-r1/Mixture-of-Thoughts[code]: 171 samplesopen-r1/Mixture-of-Thoughts[math]: 171 samplesopen-r1/Mixture-of-Thoughts[science]: 170 samplesSWE-bench/SWE-smith-trajectories(tool): 170 samples
Integration Notes
This checkpoint was generated with packed Qwen3.5/Qwen3.6 REAP support.
The packed routed expert tensors and router rows were sliced while preserving
the shared expert and the vision-language configuration. The saved model uses
the Transformers qwen3_5_moe architecture and includes tokenizer and
processor files.
Citation
@inproceedings{
lasby2026reap,
title={{REAP} the Experts: Why Pruning Prevails for One-Shot MoE compression},
author={Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=ukGxWd2aDG}
}