RivetCoder-9B-A4B
RivetCoder-9B-A4B is an experimental coding-oriented routed-expert model. It keeps
LiquidAI/LFM2.5-2.6B as a frozen 30-layer host and adds 480 frozen FFNs derived
from zai-org/GLM-5.3-Flash. Each host layer owns 16 layer-qualified candidates;
four are routed per token.
The model uses custom Transformers code and must be loaded with
trust_remote_code=True.
Model details
| Item | Value |
|---|---|
| Host | LiquidAI/LFM2.5-2.6B |
| Expert donor | zai-org/GLM-5.3-Flash |
| Router teacher | qwen/qwen3.8-27b via local LM Studio |
| Host layers | 30 |
| Candidate experts | 16 per layer, 480 total |
| Active experts | Top-4 per token |
| Folded expert shape | 2048 → 2048 → 2048 |
| Approx. total parameters | 8.74B |
| Approx. active parameters | 4.21B |
| Stored tensor bytes | 17,478,172,784 |
| Routing-control training | 60 optimizer steps |
The fixed bridge is P = [I; H] / sqrt(2), where H is a normalized,
signed, deterministically permuted order-2048 Hadamard matrix. P_out = P.T
and P.T @ P ≈ I. The bridge is folded into the expert and router weights and
is not present as a runtime module.
The GLM post-sigmoid expert-choice correction is preserved as a frozen buffer. It affects Top-K selection only; mixture weights are gathered from the uncorrected sigmoid scores.
Installation
pip install "transformers>=5.16.1,<5.17" "torch>=2.12" "accelerate>=1.13" safetensors
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HCHs/RivetCoder-9B-A4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
dtype="auto",
device_map="auto",
)
messages = [
{
"role": "user",
"content": "Implement an LRU cache in Python and include concise tests.",
}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
).to(model.device)
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.2,
do_sample=True,
)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
The BF16/mixed-FP32 checkpoint is about 16.3 GiB. A single 16 GiB GPU generally
requires CPU/disk placement through device_map="auto" or another offload
strategy.
To run the unmodified LFM host path for comparison:
model.set_coding_enabled(False)
Expert selection and router training
GLM router behavior was profiled on 4,119 coding tokens covering Python, TypeScript, Go, Rust, Java, C++, and SQL. For each sparse donor layer, 16 experts were selected using the deterministic rank score:
2 × frequency_rank + 3 × weighted_route_mass_rank
The provisional monotonic-depth mapping assigns one donor layer to every LFM layer. Selected experts and the LFM host remain frozen.
Routing controls were trained from Qwen-generated assistant responses:
- 20 coding conversations for training;
- 4 held-out coding conversations;
- 8 generic-control conversations;
- assistant-token-only causal loss through the LFM chat template;
- frozen-host self-KL and generic token-gate suppression;
- trainable tensors: router weights, token gates, and bounded residual scales.
On the small held-out set (1,417 assistant tokens), assistant CE changed from
0.605825 for the expert-off host to 0.601436 for the fused model. This is a
small internal routing check, not a standardized coding benchmark.
The frozen-host expert-off path remained bitwise identical after training.
Limitations
- This is an experimental custom architecture, not a stock LFM2 checkpoint.
- Evaluation currently consists of a very small held-out routing set; HumanEval, MBPP, SWE-bench, and broader regression results have not been reported.
- The bridge is deterministic and untrained, so donor/host representation mismatch can limit transferred expertise.
- The layer mapping is normalized-depth based rather than activation-alignment based.
- Soft token gating is enabled at inference. Hard thresholding is disabled
because the learned gates were not calibrated for a
0.5compute-skip cutoff. - CPU expert execution is substantially slower than a dedicated grouped-GEMM kernel.
Licenses and attribution
The overall checkpoint is distributed subject to the LFM Open License v1.0
in LICENSE, inherited from the LFM host. Review that license before
redistribution or commercial use.
In particular, the LFM license does not grant commercial-use rights to a legal entity whose annual revenue exceeds USD 10 million. Consult the complete license; this summary is not legal advice.
The GLM-derived expert weights originate from zai-org/GLM-5.3-Flash, whose MIT
license is included at licenses/GLM-MIT.txt.
qwen/qwen3.8-27b was used only as a sequence-level teacher for router-control
training. No Qwen model weights are included in this repository.
Detailed, path-sanitized source revisions, selection rules, hashes, folding
checks, and training metrics are available under provenance/.
Reproducibility anchors
Combined weight index SHA-256:
939b9f2bf0c48523d6a1a2ef93d987876a3e20613b33fe3715bf6b78bbc2a1bb
Routing controls SHA-256:
740f15e9bcf68efcb4d9c0f4b2bcfebff66ba199aead9cd2666156dffc60b1ad
Training run fingerprint:
68f9fa90e18e180f7037dea5e057aa296c95b59267aa09f38285363b5e517c2a