This repository holds the text branch of Apertus 1.5 8B, converted to MLX and quantized to MXFP4. It runs on Apple silicon through mlx-lm.
Apertus 1.5 is the fully open model of the Swiss AI Initiative, built at EPFL, ETH Zurich and the Swiss National Supercomputing Centre on open data.
This conversion was made independently of the Apertus release.
Model summary
The family
Six builds of the same text branch, measured on the same corpus with the same script. Perplexity is measured on the test split of Salesforce/wikitext, configuration wikitext-2-raw-v1, over 200 non-overlapping windows of 512 tokens, which is 102,400 scored tokens.
What was left out
These variants were measured on the same corpus and are not published. The table is here so that nobody repeats the work.
Three bits break this model. At a group size of 64 it answers fluently and wrongly, completing "The capital of Switzerland is" with "not a good idea"; at 32 it answers "Bern" and still reaches a perplexity of 66. Distillation recovers a large share of the loss, from 169.79 to 50.91, and the result is still five times the reference.
The two four-bit rows are here for a different reason: they work, but each is beaten by a published build that is the same size or smaller. Nothing between 4.25 and 4.5 bits per weight was worth publishing besides the two that are.
This build
MXFP4 stores each weight as a 4-bit float and shares one 8-bit power-of-two exponent across every group of 32. The format is the one mlx_lm.convert --q-mode mxfp4 writes, byte for byte, and any loader that reads MXFP4 reads this build. What changed on 2026-09-19 is which values sit inside that format.
The first version of this repository rounded every weight to the nearest representable value, with the exponent MLX picks on its own: the smallest that never clips the largest weight of the group, which on many groups throws away one of the eight levels to save a single weight. This build is calibrated instead, in three steps that each move the error somewhere cheaper:
- the exponent of every group is chosen, between the one MLX would pick and the one below it, by the error it causes on the layer's output rather than on the weights;
- GPTQ: every rounded column pushes its error onto the columns not yet rounded, in proportion to how the layer's inputs correlate on 128 windows of calibration text, and each layer is calibrated on the inputs produced by the layers already quantized;
- a Hadamard rotation of the residual stream, applied to the weights and never at run time: the output of the model is unchanged (verified at 1e-6 in float32), the architecture is unchanged, and the weights inside each group of 32 become easier to round. The RMSNorm gains are folded into the projections that read them, which is why the norm weights of this build are all 1.
Rotation alone makes things worse (perplexity 13.36 with plain rounding); it pays only together with GPTQ. Nothing of the evaluation text was used for calibration.
Its perplexity is 10.8522, +4.3% against the 8-bit build, on the protocol described above.
Beyond perplexity, on text held out from calibration, against the 8-bit build:
The multiple-choice tasks are ARC-Challenge, HellaSwag, Winogrande and Global-MMLU in English, Italian, German and French, 3,467 questions scored by log-likelihood.
# GPTQ on the MXFP4 grid, Hadamard rotation of the residual stream,
# act-order with static groups; 128 calibration windows of 512 tokens
python scripts/mxfp4_gptq.py --metodo gptq --esponente gptq \
--campioni 128 --ruota --ordine --uscita <folder>
Built on 2026-09-19 on macOS 27.0 with mlx-lm 0.31.3. The MXFP4 bytes are written by the conversion's own packer, identical to mx.quantize when given the same exponents, and every tensor was read back and checked. The perplexity above was measured with mlx 0.31.2 and again with mlx 0.32.2: the same to the fourth decimal.
Run it
pip install mlx-lm
mlx_lm.generate --model m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp4 \
--prompt "Name the capital of Switzerland and say one sentence about it." \
--max-tokens 200
From Python:
from mlx_lm import load, generate
model, tokenizer = load("m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp4")
messages = [{"role": "user", "content": "Name the capital of Switzerland."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))
Behind an OpenAI-compatible endpoint:
mlx_lm.server --model m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp4
In oMLX, search for m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp4 and download
it. Deliberation is read natively: the model opens it with <think> and
closes it with </think>, which is what oMLX looks for.
What the text branch is
The upstream release is multimodal. It reads images and audio as well as text,
and its architecture is Apertus1p5ForConditionalGeneration, which neither
mlx-lm 0.31.3 nor mlx-vlm 0.6.17 implements.
This repository holds the text decoder on its own, declared as
ApertusForCausalLM so that mlx-lm loads it. The token embedding goes from
266752 rows to 131072. The rows that go are the image and audio codebooks,
which start at index 131272 and 262344 in the upstream vocabulary. The
upstream output_vocab_size is already 131072, so the language modelling head
is untouched and no text token is lost.
The model reads and writes text. It does not accept images or audio. The whole model, towers included, is a different family: Apertus v1.5 8B MLX.
Limitations
The quantization inherits every limitation of the upstream model, which its model card describes. No output filter ships with these weights.
Quality here is measured by perplexity on one English corpus. Apertus 1.5 is multilingual, and the effect of quantization on languages other than English is not measured in this collection. Instruction following, reasoning and tool use are also unmeasured.
License and acceptable use
The weights stay under the Apache 2.0 license of the upstream release. Use is also subject to the Apertus 1.5 acceptable use policy and privacy policy:
For removal of personal or copyrighted data, write to the Swiss AI Initiative at llm-privacy-requests@swiss-ai.org or llm-copyright-requests@swiss-ai.org.
Credits
The model is the work of the Swiss AI Initiative. This repository adds the MLX conversion, the quantization and the measurements above.
@misc{ApertusV15,
author = {{Swiss AI Initiative}},
title = {Apertus v1.5},
year = {2026},
howpublished = {\url{https://huggingface.co/swiss-ai/Apertus-v1.5-8B}},
note = {EPFL, ETH Zurich, and the Swiss National Supercomputing Centre}
}