antares-1b-mlx-bf16
Unquantized bfloat16 MLX build of fdtn-ai/antares-1b,
Cisco Foundation AI's file-level vulnerability-localization model.
No quantization, no re-training, no weight surgery: every retained tensor is bit-for-bit the
original checkpoint. This repo exists only because the upstream checkpoint does not load in
mlx-lm as published.
Why this repo exists
mlx_lm.load("fdtn-ai/antares-1b") fails twice:
-
Missing
rope_theta. The upstreamconfig.json(exported with Transformers 5.x) carries the RoPE base only inside the nestedrope_parametersblock.mlx_lm'sgranitemoehybrid.ModelArgsrequires a top-levelrope_theta, which is a required field with no default — so construction fails. Here the top-level key is present with the correct value (10000000). This matters for quality, not just loading: silently falling back to a default base would wreck long-context attention. -
Duplicate
lm_head.weight.ValueError: Received 1 parameters not in model: lm_head.weight.The config sets
tie_word_embeddings: true, so the MLX model has no separatelm_headmodule, but the checkpoint still ships the tensor. Transformers ties it to the embedding at load time;mlx-lmloads weights strictly and rejects the extra key.Before dropping it, the tensor was compared against
model.embed_tokens.weightover all 100,352 rows as rawuint16— bitwise identical, confirming the weights are genuinely tied and nothing is lost. Transformers agrees: it reportslm_head.weight.data_ptr() == model.embed_tokens.weight.data_ptr().Dropping the duplicate also shrinks the checkpoint from 3.7 GB to 3.3 GB.
Everything else — tokenizer.json, tokenizer_config.json, generation_config.json,
chat_template.jinja — is copied unchanged from upstream.
Verification
Logits were compared against the Transformers reference implementation
(GraniteMoeHybridForCausalLM, transformers==5.14.1, CPU) on a 1,020-token prompt,
across all 1,020 positions and the full 100,352-token vocabulary:
| Comparison | top-1 agreement | mean abs Δlogit | mean KL |
|---|---|---|---|
| MLX float32 vs HF float32 | 99.31 % | 0.0296 | 2.70e-04 |
| MLX bfloat16 vs HF float32 | 97.26 % | 0.2361 | 1.50e-02 |
| HF bfloat16 vs HF float32 (bf16 noise floor) | 96.37 % | 0.2338 | 1.41e-02 |
Read the last two rows together: this build's deviation from the float32 reference is no larger than what Transformers' own bfloat16 path produces. At matched precision (float32 row) the two implementations agree to within floating-point accumulation order. There is no systematic error.
Usage
pip install mlx-lm
mlx_lm.generate --model dkavocado/antares-1b-mlx-bf16 --prompt "def add(a, b):" --ignore-chat-template
Roughly 80 tok/s on an M5 Pro, ~3.3 GB peak memory.
Serving for Antares CLI
Antares CLI talks to a streaming OpenAI-compatible
POST /v1/completions endpoint and builds the Granite prompt itself, so it needs the raw
completion text including the <tool_call>…</tool_call> blocks.
mlx_lm.server infers a json_tools tool-call parser from chat_template.jinja and strips those
blocks out of the text field. Antares then sees an empty answer and every scan ends as
Model ended without an explicit final submission with zero tool calls. Override the template
with a neutral one to disable that parser:
mlx_lm.server --model dkavocado/antares-1b-mlx-bf16 \
--host 127.0.0.1 --port 8001 \
--decode-concurrency 8 \
--chat-template "{% for m in messages %}{{ m['content'] }}{% endfor %}"
Antares profile (~/.antares/profiles.toml):
[profiles.mlx]
model = "dkavocado/antares-1b-mlx-bf16"
backend = "remote"
endpoint = "http://127.0.0.1:8001/v1/completions"
context_window = 32768
[profiles.mlx.generation]
max_tokens = 8192
temperature = 0.3
top_p = 1.0
frequency_penalty = 0.3
stop_tokens = ["<|end_of_text|>", "<|start_of_role|>"]
use_completions_api = true
mlx_lm.server resolves the request's model field as a path or repo id, so the profile's
model value must be the same string the server was started with (or default_model).
Model behavior and limits
Unchanged from upstream: the model localizes vulnerabilities at file level — it returns ranked candidate file paths for a given CWE. It does not give line numbers, code snippets, or fixes, and results vary between identical runs. Treat every result as a lead to verify.
License and attribution
Apache 2.0, inherited from the original model. The weights are the work of
Cisco Foundation AI — see fdtn-ai/antares-1b
for the authoritative model card, evaluation results, and intended-use guidance. The upstream
repository is gated behind an acceptance click; please read its terms there.
This repository redistributes those weights in MLX layout for Apple Silicon users, with no modification beyond the two loader fixes described above.