xlm-roberta-large-xnli-anli · MLX Swift 8-bit
Multilingual natural-language inference (entail / neutral / contradict) on Apple Silicon, at 0.63 GB.
| Base | vicgalle/xlm-roberta-large-xnli-anli @ 85981da8 |
| Task | 3-way NLI: entailment / neutral / contradiction |
| Format | MLX Swift safetensors, 8-bit affine, group size 64 (Linear + Embedding) |
| Size | 0.63 GB (fp32 original ≈ 2.2 GB) |
| Accuracy | byte-identical downstream results vs fp32 on a multilingual benchmark (below) |
| License | MIT (unchanged) |
Intended use
Detecting whether a new statement contradicts, confirms or is unrelated to an earlier one, and zero-shot classification, across languages, fully on-device. Feed (premise, hypothesis) as a sentence pair; label order follows config.json (id2label).
How it was made
The original fp32 weights were loaded with an MLX Swift BertModel (MLXEmbedders naming, after its sanitize step) and
quantized in memory with MLXNN.quantize (affine, 8-bit, group size 64, default Linear and Embedding layers). The
quantized module parameters were saved unchanged as model.safetensors; quantization.json records the scheme.
Verification
An item-linking benchmark of four multilingual packs (English, Czech and a multilingual parliament set) uses this model together with two companion models from the same family of repos (embedder, reranker, NLI). It produced byte-identical outputs with these 8-bit files and with the original fp32 weights.
Loading (read this first)
These are MLX Swift weights, not a Transformers checkpoint: transformers and sentence-transformers cannot load
them. Use the original repo for those. To load in MLX Swift:
- Build the
MLXEmbeddersBertModelfromconfig.json(with the sequence-classification head). - Call
quantize(model: model, groupSize: 64, bits: 8, mode: .affine)on it before loading weights. Quantized layers storeweight/scales/biases. - Load
model.safetensorsinto the module and use the tokenizer files as usual.
quantization.json ({"bits": 8, "groupSize": 64, "mode": "affine"}) is the marker a loader can check to take step 2.
Files
| file | content |
|---|---|
model.safetensors | 8-bit parameters |
quantization.json | quantization scheme |
config.json | copied from the source revision |
tokenizer.json, tokenizer_config.json, special_tokens_map.json | the source repo ships no tokenizer.json; all XLM-RoBERTa-large models share one SentencePiece vocabulary, so these come from BAAI/bge-reranker-v2-m3 @ 953dc6f6 (Apache-2.0) |
License and attribution
MIT, the same as vicgalle/xlm-roberta-large-xnli-anli, itself fine-tuned from FacebookAI/xlm-roberta-large (MIT). All credit for the model goes to its authors. The tokenizer files come from BAAI/bge-reranker-v2-m3 (Apache-2.0). The change here is quantization only.