Millie-1.1-35B-A3B-Ternary
Millie 1.1 is a language and vision model from LLMs For All, built for local inference on memory-constrained devices. It has approximately 35 billion total parameters and 3 billion active parameters per token. Roughly 90% of its weights use a ternary representation; the remaining tensors use higher precision.
These are the weights used by the flagship model in the initial release of the Millie iPhone app.
Files
| File | Purpose | Download size |
|---|---|---|
Millie-1.1-35B-A3B-Ternary.gguf | Language model, tokenizer, and chat template | 8.31 GB |
Millie-1.1-35B-A3B-vision.gguf | Matching vision encoder/projector for image input | 0.30 GB |
inference-settings.json | Thinking and non-thinking sampling defaults | — |
chat_template.jinja | A readable copy of the template embedded in the GGUF | — |
SHA256SUMS | Checksums for the released GGUF files | — |
Sizes are decimal GB. Disk size is not runtime RAM usage. The vision file is optional for text-only use.
Run Millie
- Download the archive for your platform from Millie Runtime v0.1.0 and extract it.
- Download
Millie-1.1-35B-A3B-Ternary.gguffrom this repository. - Open a terminal in the extracted runtime folder and run:
./bin/llama-cli \
-m /path/to/Millie-1.1-35B-A3B-Ternary.gguf \
--jinja --reasoning off \
--temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 \
--presence-penalty 1.5 --repeat-penalty 1
Replace /path/to/ with the folder where you downloaded the model, then type a message to chat.
For images, also download Millie-1.1-35B-A3B-vision.gguf and add:
--mmproj /path/to/Millie-1.1-35B-A3B-vision.gguf --image /path/to/photo.jpg
The runtime supports ordinary inference and a low-memory mode that keeps a bounded number of experts resident per layer. Low-memory mode currently requires Apple Metal. See low-memory setup.
Generation settings
Millie inherits Qwen's sampling defaults for non-thinking and thinking generation:
| Mode | Temperature | Top-p | Top-k |
|---|---|---|---|
| Non-thinking | 0.7 | 0.8 | 20 |
| Thinking | 1.0 | 0.95 | 20 |
Both use min-p 0, presence penalty 1.5, and repetition penalty 1.0. For thinking, use --reasoning on --temp 1.0 --top-p 0.95 instead.
Evaluation
We compare the Qwen 3.6 teacher, the compressed model with unrestricted expert routing, and the compressed model in a low-memory configuration.
| Benchmark | Qwen 3.6 35B-A3B | Millie unrestricted | Millie low-memory |
|---|---|---|---|
| MMLU Redux, clean question set | 93.94% | 91.62% | 91.68% |
| GPQA Diamond | 86.36% | 75.76% | 74.75% |
| HumanEval+ | 93.29% | 90.24% | 91.46% |
| IFBench, loose | 68.33% | 62.00% | 60.33% |
| IFBench, strict | 63.67% | 60.00% | 59.00% |
| MuSR | 66.93% | 64.15% | 66.40% |
The low-memory configuration uses 64 hot experts per layer, a +1 routing-logit bias for hot experts, and a simulated 8-bit (Q8_0) KV cache. The evaluations used single sampled runs with up to 64K generated tokens per example.
Attribution and license
Millie is based on Qwen3.6-35B-A3B. The model files declare Apache 2.0, matching the upstream license; a copy is included in this package. LLMs For All modified and compressed the model and provides the corresponding runtime support. The runtime is distributed separately under its MIT license.