Nemotron-3-Nano-30B-A3B — REAP-pruned to 64 experts
A 50% expert-pruned version of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B,
produced with REAP (Router-weighted Expert Activation Pruning). Half the
routed experts in each MoE layer were removed, shrinking the model from
~59 GB to ~32 GB while keeping the same routing width.
| Base | This model | |
|---|---|---|
| Routed experts / MoE layer | 128 | 64 |
| Experts activated per token | 6 | 6 (unchanged) |
| Total params | ~31.6 B | ~31.6 B nominal (fewer experts stored) |
| Size (BF16) | ~59 GB | ~32 GB |
| Architecture | nemotron_h (hybrid Mamba-Transformer MoE) | same |
What is REAP?
REAP is a one-shot, post-training
method (no retraining) that scores every expert by the mean of
router_weight × activation_norm over a calibration set, then removes the
least-salient experts. Because only a few experts fire per token, many are
redundant — pruning them reduces the model's memory footprint. Note that
experts-per-token is unchanged (6), so the active compute per token is the
same; the win is total size / memory, not decode speed.
How this was made
- Method: REAP, layer-wise (block-by-block) calibration.
- Compression:
compression_ratio 0.5→ 64 of 128 routed experts kept per MoE layer. - Calibration: 256 samples from
theblackcat102/evol-codealpaca-v1at 512-token context. - Hardware: pruned on an NVIDIA DGX Spark (GB10).
Caveats
- This is a moderate-calibration prune (256 samples). It generates coherent, on-task text in sanity checks, but has not been run through formal benchmarks — expect some quality regression vs the base model, especially outside the coding/reasoning calibration domain. A production prune would use a larger, more diverse calibration set.
nemotron_hincludes Mamba layers. Withtrust_remote_code=False(native transformers modeling), it runs on a pure-torch fallback; installmamba-ssm+causal-conv1dfor the fast path.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
repo = "sapidlabs/Nemotron-3-Nano-30B-A3B-REAP-64e" # this repo
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="cuda"
).eval()
msgs = [{"role": "user", "content": "Write a Python function for the nth Fibonacci number."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
License
Inherits the base model's license (NVIDIA Open Model License). This is a derivative of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B.