Kimi-K3-Audit-240E-MLX-MXFP4

Kimi-K3-Audit-240E is Kimi K3 specialised for security auditing. It keeps 240 of the 896 routed experts in each layer, which brings the model onto a single 512 GB Apple silicon machine. Expert selection was calibrated on kernel source and audit traces, targeting domain focus rather than compression.

For a pair of 512 GB machines, the larger Kimi-K3-Audit-451E build measures at parity with the full release.

Intended use

Security auditing and analysis of source code, with tool use. General conversation, translation and non-technical writing are out of scope, and a general-purpose model should be used for them.

Across eight code corpora in five languages, pruning changes perplexity least on C and C++, ranging from 1.05× on Linux to 1.38× on Python.

Evaluation

Measured against the original MXFP4 release (896E) on identical tokens. KLD is KL(896E ‖ 240E) over the full output distribution.

Audit traces

Each of 24 kernel-audit sessions contributes one 1025-token window, rendered through the model's own chat template so the structure matches deployment.

Tool-call spans diverge least of any span measured. 78% of tool-call tokens fall below 0.01 divergence, against 16% of thinking tokens.

Code and prose corpora

Build comparison

Usage

This model requires Kimi K3 support from mlx-lm PR #1626, which has not yet been merged. Until it lands in an mlx-lm release, install from the PR branch.

pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1626/head
pip install tiktoken

from mlx_lm import load, generate

model, tokenizer = load(
    "catalystsec/Kimi-K3-Audit-240E-MLX-MXFP4",
    tokenizer_config={"trust_remote_code": True},
    trust_remote_code=True,
)

prompt = "Audit do_wp_page() in mm/memory.c for memory-safety issues."
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, verbose=True)

Prefill step size

Prefill memory scales with chunk × context and is separate from the weights, so long contexts need a smaller step size than the default. Maximum context on a 512 GB M3 Ultra at batch size 1:

mlx_lm.server --model catalystsec/Kimi-K3-Audit-240E-MLX-MXFP4 \
              --prefill-step-size 512

Throughput

Batch size 1 on 512 GB machines at the default step size. The two-machine figures use tensor parallelism over Thunderbolt RDMA, with peak memory per node.

1× M3 Ultra

2× M3 Ultra

Generation is bandwidth-bound and changes little with context, since the latent KV cache is small next to the weights read per token. A second machine gives 1.5× generation and 1.8× prefill.

Sampling follows the base model, temperature 1.0 and top-p 0.95.

Limitations

Evaluation is teacher-forced throughout and measures next-token prediction on reference text. Generation-time behaviour, including loop rate and stop-token reliability, was not evaluated.

Standard downstream benchmarks were not evaluated. Pruning preserves next-token performance less well on Rust and Python than on C, C++ and Swift.

License

Inherits the Kimi K3 license from the base model.