About
- Update(1)
2026-04-24: apparently, VirusTotal does not like the llamafiles and marks them "suspicious". There's not much I can do there, I guess. Let us wait untill the other scanners try it; until then -- feel free to e.g. ClamAV scan them after download, but also feel free to redo it yourself following the links provided: your result will be binary compatible with what is uploaded here. - Update(2)
2026-04-24: hmm, the "suspicious" flag seems to be cleared on some binaries. Good. May be just give it time to update the databases. - Update(3)
2026-04-24: Ok, I uploaded some components under test/ -- let us see the scan verdict on these. Also, I discovered that VirusTotal gives a very clear false positive onzipalign-0.10.0, which is uploaded at virustotal.false_alarm/. Needless to say, all the executables are coming from Mozilla' GitHub repository Releases page, and are exactly the same -- see virustotal.false_alarm/README.md- Also, apparently only one of all those 75 engines (named "SecureAge") marks this binary as "malicious", and its comment says "Acronis (Static ML)" -- whatever that means. Well, the binary does have an embedded LLM (which is like the whole point here), so may be this is what causes this misunderstanding.
A llamafile is a universal (APE) single-file executable, which is based on llama.cpp-built binaries. It contains an embedded LLM model and can provide a console and/or a web interface to chat with it.
- "and/or" here has a literal meaning -- one can either run a console chat (
--chat), or a web server (--server), or -- by default -- both. - NB(1): by default the web server attempts to listen on all available interfaces -- run it as
--host 127.0.0.1to make it private to your system only! - NB(2): there's also a cli completion interface (
--cli), if you wish -- check the docs!
How to use:
- https://github.com/mozilla-ai/llamafile/blob/main/docs/running_llamafile.md
- https://github.com/mozilla-ai/llamafile/blob/main/docs/quickstart.md
Building references:
- https://github.com/mozilla-ai/llamafile/blob/main/docs/creating_llamafiles.md
- https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/blob/main/mmproj-BF16.gguf
- https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/blob/main/Qwen3.5-4B-UD-Q4_K_XL.gguf
- https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/blob/main/Qwen3.5-4B-IQ4_XS.gguf
Default arguments:
temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
// recommended (unsloth) for coding in thinking mode:
--ctx-size 16384 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.00 \
--chat-template-kwargs '{"enable_thinking":true}'
// recommended (unsloth) for reasoning tasks:
--ctx-size 16384 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
To enable "thinking mode" (slower but better results):
--chat-template-kwargs '{"enable_thinking":true}'
Extra recommendations -- try the following if generation falls into an endless loop:
- https://unsloth.ai/docs/models/qwen3.5 (same source)
presence_penalty = 0.0 to 2.0
Quirks:
- https://github.com/mozilla-ai/llamafile/issues/373
- https://github.com/mozilla-ai/llamafile/tree/0.9.0?tab=readme-ov-file#gotchas-and-troubleshooting
For some quirks and workarounds (like how to make this run on WSL), see the second link above; in particular, if you are seeing a message like this:
Cannot open assembly '...': File does not contain a valid CIL image.
The reason might be that you have wine installed and it tries to run the executable (and fails). There are workarounds available -- see above; if that does not work, please open an issue, and when I see it (which might not happen immediately), we will think of something.