LFM2.5-8B-A1B-REAP-50
Model Description
This is a 50% expert-pruned variant of LiquidAI/LFM2.5-8B-A1B, produced by REAP (Ranking and Eliminating Experts via Activation Patterns) pruning.
- Base: LFM2.5-8B-A1B — 32 experts per MoE layer, 8.47B params, 16.94 GB (bf16)
- This model: 16 experts per MoE layer, 4.59B params, 8.57 GB (bf16)
- Compression: 45.5% size reduction, 50% expert removal
- No fine-tuning: REAP produces an exact subset of original expert weights — gate/router weights and surviving expert weights are bit-for-bit identical to the base model.
How REAP Works
REAP scores each expert's importance using activation patterns from a calibration set, then prunes the lowest-importance experts per-layer independently. This is not uniform pruning — different experts survive in different layers:
- Expert 27 survives in 73% of layers (most important)
- Expert 29 survives in 32% of layers (most pruned)
Calibration: 200 samples from evol-codealpaca-v1
Tool: reap-cuda
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"konic-labs/LFM2.5-8B-A1B-REAP-50",
trust_remote_code=True,
dtype=torch.bfloat16,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
"konic-labs/LFM2.5-8B-A1B-REAP-50", trust_remote_code=True
)
messages = [{"role": "user", "content": "What is 2+2?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False, temperature=None)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Works with vLLM out of the box (native Lfm2MoeForCausalLM support).
Benchmark Results
| Benchmark | Base (published) | This model (REAP-50) | Retention |
|---|---|---|---|
| MATH500 | 88.76% | 77.0% | 86.7% |
| BFCLv3 (single-turn) | 64.79% | 59.07% | 91.1% |
MATH500 per-difficulty
| Level | Accuracy |
|---|---|
| L1 | 90.7% |
| L2 | 84.4% |
| L3 | 81.9% |
| L4 | 82.8% |
| L5 | 58.2% |
BFCLv3 per-category
| Category | Accuracy |
|---|---|
| simple_python | 78.50% |
| multiple | 85.00% |
| parallel | 70.00% |
| parallel_multiple | 70.50% |
| irrelevance | 80.00% |
| live_simple | 60.47% |
| live_multiple | 54.32% |
| live_parallel | 31.25% |
| live_parallel_multiple | 54.17% |
| live_irrelevance | 75.90% |
| live_relevance | 68.75% |
Throughput (vLLM, NVIDIA L4)
| Config | Tokens/sec |
|---|---|
| Offline batch (500 prompts) | 1,280 |
| HTTP server (conc=256) | 2,282 |
Intended Use
Research and deployment of compressed MoE models. This is the intermediate pruning stage — apply your own quantization (AWQ, GPTQ, etc.) on top, or use directly in bf16.
Citation
@software{reap-cuda,
title = {REAP: Ranking and Eliminating Experts via Activation Patterns},
author = {Konic Labs},
url = {https://github.com/egesabanci/reap-cuda}
}
License
Apache-2.0 (inherited from base model).