Apertus v1.1 Swiss multilingual retrieval embeddings
A standalone Sentence Transformers retrieval model derived from
swiss-ai/Apertus-v1.1-0.5B.
The model targets multilingual retrieval for German, English, French, Italian,
Romansh, and Alemannic / Swiss German.
Model
| Property | Value |
|---|---|
| Parameters | 439,397,928 (0.439B) |
| Source | swiss-ai/Apertus-v1.1-0.5B |
| Apertus lineage | swiss-ai/Apertus-8B-2509 |
| Transformer layers | 20 / 20 |
| Attention | bidirectional |
| Pooling | mean |
| Projection | 1024 → 1024 |
| Native embedding dimension | 1024 |
| Matryoshka dimensions | [1024, 768, 512, 256] |
| Maximum inference length | 1024 tokens |
LoRA is used during retrieval training and fully merged before publication. The released repository is therefore a standalone model without a PEFT adapter dependency.
Training
Training uses MultipleNegativesRankingLoss with cross-device negatives and
MatryoshkaLoss.
All datasets were loaded from Hugging Face at exact immutable revisions. The released
data_manifest.json records source URLs, dataset commit SHAs, the recipe signature and SHA256
hashes of the processed data files.
Training data:
wikimedia/wikipedia: monolingual Swiss-core retrieval pairs plus a small multilingual preservation replay.eljuanina/VotingBooklets-v1: DE/FR/IT/RM cross-lingual alignment.ZurichNLP/SwissGov-RSD: EN↔DE/FR/IT cross-lingual alignment.
Actual final training set: 31,483 rows (23,149 Swiss monolingual, 6,884 cross-lingual, 1,450 multilingual preservation).
Training profile: quality. Configuration: 2000 steps,
batch/GPU 16, learning rate 5e-05,
training length 512 tokens, gradient checkpointing
True, LoRA r=8, alpha=16.
Evaluation
Development evaluation uses held-out Wikipedia pairs,
eljuanina/VotingBooklets-Diamond-v1,
and a held-out SwissGov-RSD split. Diamond is evaluation-only and is therefore
not listed in the datasets training metadata above.
| Diagnostic | Dim | Accuracy@1 | Recall@10 | nDCG@10 | MRR@10 |
|---|---|---|---|---|---|
| Swiss monolingual | 1024 | 0.8042 | 0.9271 | 0.8638 | 0.8439 |
| Swiss monolingual | 512 | 0.7958 | 0.9250 | 0.8594 | 0.8386 |
| Swiss cross-lingual | 1024 | 0.2812 | 0.9437 | 0.6346 | 0.5326 |
| Swiss cross-lingual | 512 | 0.2667 | 0.9396 | 0.6242 | 0.5203 |
Monolingual development queries: 480
Cross-lingual development queries: 480
These are internal development diagnostics, not MTEB/MMTEB benchmark claims.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("andreasmartin/apertus-v1.1-swiss-embed-0.4b-bidir")
queries = model.encode_query([
"Welche Aufgaben hat der Bundesrat?"
])
documents = model.encode_document([
"Le Conseil fédéral est l'autorité exécutive suprême de la Confédération suisse.",
"Der Nationalrat ist die grosse Kammer der Bundesversammlung.",
])
scores = model.similarity(queries, documents)
print(scores)
For compact Matryoshka embeddings:
embeddings = model.encode_query(texts, truncate_dim=512)
Limitations
- Wikipedia title→paragraph pairs are pseudo-retrieval supervision.
- Parallel passages provide semantic alignment rather than natural search queries.
- Alemannic / Swiss German has monolingual but no explicit parallel supervision.
- External MTEB/MMTEB/MIRACL and Swiss/domain-specific evaluation is required for comparative quality claims.
License and attribution
Apertus is developed by the Swiss AI Initiative. This is an independent retrieval adaptation and not an official Swiss AI Initiative embedding release.
The direct source model is swiss-ai/Apertus-v1.1-0.5B. Available license and usage-policy
files from the source repository are copied into this model repository.
See the linked Hugging Face dataset cards for dataset licensing and attribution.