antares-1b-bf16-GGUF
Unquantized bfloat16 GGUF build of fdtn-ai/antares-1b,
Cisco Foundation AI's file-level vulnerability-localization model.
Every published GGUF of this model so far is quantized (Q2_K … Q8_0). This one is not: weights are the original bf16 values, converted only in container format.
| File | Size | Weights |
|---|---|---|
antares-1b-bf16.gguf | 3.42 GiB | BF16 (F32 norms), unquantized |
How it was built
python convert_hf_to_gguf.py fdtn-ai/antares-1b --outtype bf16 --outfile antares-1b-bf16.gguf
llama.cpp build 10090 (720d7fa), converter class GraniteHybridModel.
One caveat if you reproduce this: the upstream config.json (exported with Transformers 5.x) keeps
the RoPE base only inside the nested rope_parameters block. Add a top-level
"rope_theta": 10000000 before converting and then verify it landed in the GGUF —
a silent fall back to the default base would quietly wreck long-context attention:
python -c "from gguf import GGUFReader; r=GGUFReader('antares-1b-bf16.gguf'); print(r.fields['granite.rope.freq_base'].contents())"
# 10000000.0
This build's metadata was checked: rope.freq_base 10000000, attention.scale 0.0078125,
embedding_scale 12.0, residual_scale 0.22, logit_scale 8.0, block_count 40,
head_count 16, head_count_kv 4, vocab_size 100352.
Measurements
Perplexity over 9 chunks of Python source at n_ctx 2048, and llama-bench on an Apple M5 Pro
(Metal), same machine, same llama.cpp build:
| Build | PPL (lower is better) | Generation | Prompt | Size |
|---|---|---|---|---|
| BF16 (this repo) | 4.7953 ± 0.1847 | 75.5 t/s | 5420 t/s | 3.42 GiB |
| Q8_0 | 4.8648 ± 0.1894 | 124.8 t/s | 5910 t/s | 1.82 GiB |
So the honest trade-off: BF16 gives you the exact checkpoint and ~1.4 % lower perplexity, Q8_0 gives you 1.65× faster generation at half the memory. If you are running batched agentic workloads where throughput dominates, Q8_0 is the pragmatic choice; if you are validating results or care about matching the reference implementation, use this one.
Usage
llama-server -m antares-1b-bf16.gguf --alias antares-1b --host 127.0.0.1 --port 8003 \
-np 8 -c 65536 --metrics
Note that -c is the total context split across -np slots — the command above gives each of
8 slots 8192 tokens.
With Antares CLI
Antares needs a streaming OpenAI-compatible POST /v1/completions endpoint and builds the Granite
prompt itself, so it needs the raw completion text including <tool_call>…</tool_call> blocks.
llama.cpp does not touch those on the completions route, so no workaround is needed here (unlike
mlx_lm.server, which strips them via its json_tools parser).
[profiles.llama]
model = "antares-1b"
backend = "remote"
endpoint = "http://127.0.0.1:8003/v1/completions"
context_window = 32768
[profiles.llama.generation]
max_tokens = 8192
temperature = 0.3
top_p = 1.0
frequency_penalty = 0.3
stop_tokens = ["<|end_of_text|>", "<|start_of_role|>"]
use_completions_api = true
Keep context_window at or below the per-slot context, otherwise long investigations get rejected
by the server.
Related
- MLX build of the same unquantized weights, for Apple Silicon:
DKAvocadoo/antares-1b-mlx-bf16
Model behavior and limits
Unchanged from upstream: the model localizes vulnerabilities at file level — it returns ranked candidate file paths for a given CWE, without line numbers, snippets, or fixes, and results vary between identical runs. Treat every result as a lead to verify, not as proof.
License and attribution
Apache 2.0, inherited from the original model. The weights are the work of Cisco Foundation AI;
see fdtn-ai/antares-1b for the authoritative model
card, evaluations, and intended-use guidance. The upstream repository is gated behind an acceptance
click — please read its terms there. This repository only changes the container format.