Kimi-K3-Audit-451E-MLX-MXFP4
Kimi-K3-Audit-451E is Kimi K3 specialised for security auditing. It keeps 451 of the 896 routed experts in each layer, which brings the model onto two 512 GB Apple silicon machines at half the size of the release. On kernel-audit workloads it measures at parity with the full model. Expert selection was calibrated on kernel source and audit traces, targeting domain focus rather than compression.
For a single 512 GB machine, use the smaller Kimi-K3-Audit-240E build instead.
| Base | moonshotai/Kimi-K3, MXFP4 release |
| Routed experts | 451 of 896 per layer |
| Retained expert weights | bit-identical to the release |
| Size | 725 GiB, 150 shards |
| Modality | text, vision tower is retained but unevaluated |
| Loader | mlx-lm with Kimi K3 support (PR #1626) |
Intended use
Security auditing and analysis of source code, with tool use. General conversation, translation and non-technical writing are out of scope, and a general-purpose model should be used for them.
| corpus | perplexity ratio | accuracy 896E | accuracy 451E | change, pts |
|---|---|---|---|---|
| XNU kernel C | 1.01× | 80.8% | 80.2% | −0.6 |
| Linux kernel C | 1.01× | 92.3% | 92.0% | −0.3 |
| English prose | 6.95× | 92.0% | 50.6% | −41.4 |
Across eight code corpora in five languages, pruning changes perplexity by at most 1.10×, against 1.38× for the 240E build.
Evaluation
Measured against the original MXFP4 release (896E) on identical tokens. KLD is
KL(896E ‖ 451E) over the full output distribution.
Audit traces
Each of 24 kernel-audit sessions contributes one 1025-token window, rendered through the model's own chat template so the structure matches deployment.
| span | ppl 896E | ppl 451E | mean KLD | median KLD | top-1 agreement |
|---|---|---|---|---|---|
| tool call | 1.616 | 1.607 | 0.024 | 0.00001 | 97.0% |
| think | 5.874 | 6.066 | 0.068 | 0.043 | 86.6% |
| response | 3.728 | 3.560 | 0.128 | 0.008 | 90.8% |
| other | 5.044 | 5.077 | 0.207 | 0.101 | 79.7% |
| all | 4.232 | 4.185 | 0.109 | 0.023 | 88.6% |
Overall perplexity on audit traces is slightly below the release. Tool-call spans diverge least of any span measured. 81% of tool-call tokens fall below 0.01 divergence, against 24% of thinking tokens.
Code and prose corpora
| corpus | language | ppl 896E | ppl 451E | mean KLD | top-1 agreement |
|---|---|---|---|---|---|
| Linux kernel | C | 1.374 | 1.383 | 0.019 | 97.6% |
| XNU | C | 2.213 | 2.242 | 0.029 | 95.0% |
| llama.cpp | C++ | 1.664 | 1.704 | — | 95.6% |
| Swift stdlib | Swift | 1.546 | 1.597 | 0.044 | 95.4% |
| Linux rust/ | Rust | 1.596 | 1.652 | 0.053 | 94.9% |
| JavaScriptCore | C++ | 1.560 | 1.661 | 0.088 | 94.0% |
| SGLang | Python | 1.719 | 1.874 | 0.111 | 92.5% |
| V8 | C++ | 1.660 | 1.821 | 0.106 | 92.7% |
| wikitext | English | 1.402 | 9.749 | 1.929 | 52.9% |
Build comparison
| build | size | perplexity ratio | 512 GB machines |
|---|---|---|---|
| MXFP4 release, 896E | 1.56 TB | 1.00× | 4 |
| 2bit-UVMAX, 896E | 761 GiB | 1.20× | 2 |
| 451E, this model | 725 GiB | 0.99× | 2 |
| 240E | 408 GiB | 1.04× | 1 |
Usage
This model requires Kimi K3 support from mlx-lm PR #1626, which has not yet been merged. Until it lands in an mlx-lm release, install from the PR branch on both machines.
pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1626/head
pip install tiktoken
The model does not fit on one 512 GB machine. Serve it across two with tensor parallelism, using an MLX distributed hostfile for the pair. A fast interconnect matters, Thunderbolt RDMA works well.
mlx.launch --backend jaccl --hostfile hosts.json --env MLX_METAL_FAST_SYNCH=1 -- \
mlx_lm.server --model catalystsec/Kimi-K3-Audit-451E-MLX-MXFP4
Prefill memory scales with chunk × context and is separate from the weights,
so very long prompts may need a smaller --prefill-step-size than the default
2048.
Sampling follows the base model, temperature 1.0 and top-p 0.95.
Limitations
Evaluation is teacher-forced throughout and measures next-token prediction on reference text. Generation-time behaviour, including loop rate and stop-token reliability, was not evaluated.
Standard downstream benchmarks were not evaluated.
License
Inherits the Kimi K3 license from the base model.