VultronCoderAtlas
VultronCoderAtlas is a 2.10 trillion parameter sparse mixture-of-experts model derived from Kimi K3. It targets coding, agentic, tool-use, and multimodal workloads.
VultronCoderAtlas applies Router-weighted Expert Activation Pruning (REAP) to Kimi K3. Cerebras Research introduced the method in REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression.
The model retains 672 routed experts in each of Kimi K3's 92 MoE layers. It preserves top-16 routing, the native vision path, and the 1,048,576-token context limit. Its packed checkpoint is 1.199 TB, down from 1.561 TB for the source checkpoint.
Model details
| Field | VultronCoderAtlas |
|---|---|
| Base model | moonshotai/Kimi-K3 at revision 9f62e4e9fffbd0a83ddd60e1c209d828994b3569 |
| Architecture | 93 transformer blocks: 1 dense block and 92 MoE blocks |
| Attention layout | 23 MLA blocks and 69 KDA blocks |
| Routed experts per MoE layer | 672 |
| Routed experts selected per token | 16 |
| Shared experts per MoE layer | 2 |
| Total parameters | 2.10 T |
| Active parameters per token | Approximately 107 B |
| Expert weight format | Native MXFP4, 4-bit, group size 32 |
| Packed checkpoint size | 1,199,059,386,443 bytes |
| Context limit | 1,048,576 tokens |
| Input modalities | Text and image |
| Primary output | Text |
The total and active parameter counts are engineering estimates.
Construction
REAP scores each routed expert by averaging its post-renormalization router weight multiplied by its output norm over the tokens routed to it. Atlas measured these signals through Kimi K3's packed MXFP4 serving path using a decontaminated calibration mixture of code, agentic and tool-use trajectories, reasoning, technical prose, and vision.
Each MoE layer retains its 672 highest-scoring experts. The checkpoint compacts their packed tensors and permutes the corresponding router gate and correction-bias rows into the same order. Shared experts, attention, embeddings, and the vision tower preserve their source tensors byte-for-byte.
Deterministic half-splits of the calibration corpus produced a median per-layer Spearman correlation of 0.971, a fifth-percentile correlation of 0.801, a minimum sampling density of 0.981, and mean pooled saliency retention of 0.860.
Intended use
VultronCoderAtlas is intended for:
- repository-scale coding and software maintenance;
- coding agents that use shell, file, and structured tools;
- long-context technical analysis;
- image-and-text tasks supported by the retained Kimi K3 vision path;
- OpenAI-compatible inference with preserved reasoning history.
The model always uses thinking. Multi-turn clients must pass the complete prior assistant message back to the model, including reasoning_content, content, and tool_calls. Dropping the reasoning history changes the protocol used for release evaluation.
Evaluation
The release evaluation used the immutable 672-expert checkpoint. Speculative decoding was disabled for the final DeepSWE, Terminal-Bench 2.1, HumanEval+, MBPP+, and MMMU-Pro results. Prefix caching was enabled for DeepSWE, Terminal-Bench 2.1, HumanEval+, and MBPP+.
Local-harness rows run Kimi K3 and VultronCoderAtlas with the same task revision and protocol. Moonshot-published rows report Kimi K3 values from the pinned upstream model card.
| Benchmark | Comparison basis | Kimi K3 | VultronCoderAtlas |
|---|---|---|---|
| DeepSWE v1.1 | Moonshot published | 67.5 | 64.6 |
| MMMU-Pro, no Python, max | Moonshot published | 81.6 | 79.5 |
| Terminal-Bench 2.1 | Local Kimi Code Harness | 76.4 | 75.3 |
| HumanEval+ | Local Kimi Code Harness | 78.7 | 82.9 |
| MBPP+ | Local Kimi Code Harness | 74.3 | 76.5 |
| SWE-bench Verified | Local Kimi Code Harness | 93.8 | 95.2 |
| LiveCodeBench v6 | Local Kimi Code Harness | 91.1 | 92.1 |
| BFCL v4 multi-turn | Local Kimi Code Harness | 63.8 | 63.4 |
Deployment
The checkpoint requires an inference engine with Kimi K3 architecture support, native MXFP4 kernels, and Kimi K3 reasoning and tool-call parsers. Release qualification used vLLM development builds with tensor parallelism across eight GPUs.
The following command shows the common vLLM interface. Engine version, image, KV-cache allocation, attention backend, and GPU-specific flags must be pinned for the target hardware before production use.
vllm serve vultr/VultronCoderAtlas-Kimi-K3-2T \
--served-model-name VultronCoderAtlas \
--trust-remote-code \
--tensor-parallel-size 8 \
--max-model-len 1048576 \
--reasoning-parser kimi_k3 \
--enable-auto-tool-choice \
--tool-call-parser kimi_k3
Qualify the requested context length against available KV-cache memory, concurrency, image inputs, output budget, and the selected KV-cache dtype.
OpenAI-compatible client
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="unused",
)
messages = [
{
"role": "user",
"content": "Inspect this repository and identify the cause of the failing test.",
}
]
response = client.chat.completions.create(
model="VultronCoderAtlas",
messages=messages,
reasoning_effort="max",
max_tokens=4096,
)
assistant_message = response.choices[0].message
messages.append(assistant_message.model_dump(exclude_none=True))
Preserve the complete assistant_message in subsequent requests.
Limitations
- The packed checkpoint is approximately 1.2 TB. Deployment requires multi-GPU hardware and a Kimi K3-capable inference stack.
- The one-million-token value is the model context limit. Effective request length depends on KV-cache allocation, concurrency, image inputs, and output budget.
- Agentic evaluations show trial variance. Reported means and trial counts are included where available.
- Release qualification covers English workloads.
- The model can emit incorrect code or invalid tool calls. It can leave fixes incomplete. Run generated changes through the repository's tests and review process.
License
The release uses the Kimi K3 License. Distributed copies and substantial portions of the model must include the upstream copyright and permission notices.