Laguna-S-2.1 · Winnow keep-75
poolside/Laguna-S-2.1 (118B-A8B) with 25% of routed-expert channels removed by winnow variable-width expert pruning — 89.2B parameters, no healing/retraining.
Instead of deleting whole experts (REAP-style), winnow shrinks each expert's
SwiGLU intermediate width according to a per-channel importance score
(router_weight × |post-SwiGLU activation| × down-projection column L2),
allocated by a per-layer global rank. 97.7% of experts survive at reduced,
ragged widths; the shared expert, attention, and embeddings are untouched.
Scoring, pruning, and evaluation all ran layer-streamed on 3×RTX 3090 (the source model never fit in memory): one decoder layer resident per GPU, residual streams ping-ponged through disk, calibration data-parallel across GPUs. Calibration: 2.1M tokens of bigcode/the-stack-smol.
Results (unhealed)
| model | params | held-out code ppl | HumanEval pass@1 |
|---|---|---|---|
| Laguna-S-2.1 (base) | 118B | 2.947 | — |
| keep-75 (this repo) | 89.2B | 2.980 (+1.1%) | — |
| keep-50 (here) | 60.8B | 3.399 (+15.3%) | 89.6% |
- Perplexity: 64 held-out the-stack-smol sequences (2048 tokens each), identical ruler for all three rows, computed layer-streamed.
- HumanEval: greedy, chat-extract protocol, thinking disabled, max 1280 new tokens, INT8-expert serving. Not comparable to raw-completion HumanEval numbers or to poolside's published agentic benchmarks (SWE-bench, Terminal-Bench 2.1), which use a different harness and far larger budgets.
- Qualitative agentic check: with a 16k serving context the model autonomously solved several classic Terminal-Bench 1.0 core tasks end-to-end on the same 3×3090 host; most failures were context exhaustion, not task competence.
Serving
At ~95GB INT8 this model does NOT fit on 3×24GB GPUs; it needs ~2×80GB (or 4×40GB) cards, via the winnow vLLM plugin (ragged experts are not representable in stock FusedMoE):
vllm serve hbfreed/Laguna-S-2.1-winnow-keep50 \
--enforce-eager --pipeline-parallel-size 3 --quantization experts_int8 \
--max-model-len 16384
Measured on 3×3090: 34 tok/s single-stream, ~150 tok/s at batch 8. Tensor parallelism is unsupported (packed ragged weights don't shard); pipeline parallelism is the intended mode.
Reference (unfused) loading works with plain transformers ≥ 5.13:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"hbfreed/Laguna-S-2.1-winnow-keep50", trust_remote_code=True)
winnow.json records the full pruning plan, calibration provenance, and
per-expert channel indices.
Pruned with winnow's stream-prune; see the winnow repository for the method
writeup. Base model by poolside, released under
OpenMDW-1.1; this derivative keeps that license.