elephant-embeddings-v1-text-small

elephant-embeddings-v1-text-small is a multilingual long-context embedding model purpose-built for agent-native retrieval, memory, and decision workflows.

It is designed for systems where embeddings sit on the semantic hot path rather than at the edge of the stack: memory lookup, knowledge retrieval, tool matching, task routing, long-horizon recall, clustering, and multilingual indexing. Its value is not just a benchmark score, but a practical operating profile that fits real agent runtimes: 32K context, 2D Matryoshka adaptability across dimensions and layers, 307M-class deployability, and strong latency-quality efficiency under repeated inference.

In short, elephant-embeddings-v1-text-small is built for teams that want one embedding space to support fast routing, scalable retrieval, and high-confidence semantic matching without paying the operational cost of a much larger model.

Why it fits agentic workloads

Agentic systems do not call embeddings once. They call them everywhere: before retrieval, during routing, when matching tools, when searching memory, and while compressing or re-ranking state. That means a useful agent embedding model must be more than accurate — it must also be flexible under tight runtime budgets.

elephant-embeddings-v1-text-small is designed around that reality.

1. One model, many budget tiers

This model supports Matryoshka embeddings, which means you can encode once at full size and truncate to smaller dimensions with limited quality loss.

That is especially useful for agent systems because different stages of the stack often need different budgets:

• 64d for very cheap candidate generation, broad routing, or huge memory banks
• 256d for balanced retrieval over large corpora
• 768d for highest-quality retrieval, offline indexing, or final-stage matching

Instead of managing separate embedding models for each tier, you can keep one semantic space and choose the dimensional budget that matches the task.

2. 2D Matryoshka gives runtime flexibility, not just storage savings

The model is trained with 2D Matryoshka behavior:

• dimension reduction for smaller vectors and lower storage / bandwidth cost
• layer reduction for lower-latency inference paths in custom runtimes

This matters for agents because the same system often mixes:

• latency-sensitive routing decisions
• high-volume memory scans
• higher-quality retrieval for final evidence gathering

A single model that can serve multiple latency / quality profiles is much easier to operate than a stack of unrelated specialized encoders.

3. Long context helps when agent state is not naturally short

Many agent workloads are not short isolated queries. They involve:

• tool descriptions
• execution traces
• long notes
• merged memory summaries
• multi-hop research snippets
• large document chunks

With 32,768 tokens of context length, elephant-embeddings-v1-text-small can represent larger semantic units before you are forced into aggressive chunking. That helps preserve cross-section meaning in long documents and richer memory entries.

4. Small enough to be operationally practical

At roughly 307M parameters, this model sits in a useful middle ground:

• substantially lighter than large embedding models in the 600M+ or multi-billion range
• still expressive enough for multilingual retrieval and similarity work
• easier to host in systems where embedding is part of a hot path rather than an occasional offline batch

For agentic platforms, that usually means better economics and simpler scaling.

5. One embedding space across the stack

Agent systems are easier to operate when routing, retrieval, memory search, and semantic matching all live in the same vector space.

elephant-embeddings-v1-text-small is well suited to that pattern:

• 64d can serve broad routing and large-memory scanning
• 256d can cover the main retrieval tier
• 768d can stay reserved for the highest-fidelity matching paths

That means one model can cover multiple semantic stages without forcing the system to juggle incompatible encoders, duplicated indexes, or divergent retrieval behavior.

Model at a glance

Headline results

These numbers make the model particularly attractive for systems that must balance quality, latency, vector size, and deployment simplicity instead of optimizing only for leaderboard peak score.

Usage

Basic usage with Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("/path/to/elephant-embeddings-v1-text-small")

texts = [
    "Find tool descriptions related to browser automation.",
    "检索和用户历史偏好相关的记忆。",
    "Retrieve notes about deployment failures in staging.",
]

embeddings = model.encode(texts)
print(embeddings.shape)  # (3, 768)

Matryoshka truncation for smaller vectors

import torch.nn.functional as F
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("/path/to/elephant-embeddings-v1-text-small")
embeddings = model.encode(texts, convert_to_tensor=True)

# Balanced retrieval tier
embeddings_256d = F.normalize(embeddings[:, :256], p=2, dim=1)

# Ultra-cheap routing / large memory-bank tier
embeddings_64d = F.normalize(embeddings[:, :64], p=2, dim=1)

Long-context encoding

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("/path/to/elephant-embeddings-v1-text-small")
model.max_seq_length = 8192  # can be increased up to 32768

long_note = "..."
embedding = model.encode(long_note)

Why Matryoshka matters for agents

A common agent stack has several retrieval-like stages:

1. broad candidate fetch over a very large store
2. narrower semantic lookup over a smaller candidate set
3. high-confidence final matching before action or answer synthesis

Matryoshka lets one model support all three stages:

That is often a better operational story than mixing several incompatible embedding models across the same pipeline.

Evaluation details

MTEB benchmark (24 tasks)

STS benchmark comparison

2D Matryoshka quality matrix (STS)

Long-context retrieval (4K tokens)

Throughput (AMD MI300X)

Training

Data

Trained on BAAI/bge-m3-data with multilingual triplets across diverse domains.

Configuration

• Base model: llm-semantic-router/mmbert-32k-yarn
• Loss: Matryoshka2dLoss (combines adaptive layer loss and Matryoshka loss)
• Matryoshka dimensions: [768, 512, 256, 128, 64]
• Max sequence length: 32768
• Batch size: 16 (effective 32 with gradient accumulation)
• Learning rate: 2e-5
• Hardware: AMD Instinct MI300X

Recommended use cases

elephant-embeddings-v1-text-small is especially well suited for:

• Agent memory retrieval across long, mixed-format notes or histories
• Tool and skill selection where descriptions need semantic matching
• Knowledge-base retrieval for assistants and RAG systems
• Multilingual search across mixed-language corpora
• Large memory banks that benefit from 64d / 256d vector tiers
• Long-document semantic indexing where short-context encoders lose structure

Model lineage and packaging

elephant-embeddings-v1-text-small is derived from llm-semantic-router/mmbert-embed-32k-2d-matryoshka and distributed here as a lean Sentence Transformers / PyTorch package.

This build intentionally omits bundled ONNX artifacts so the model remains smaller and easier to move, mirror, cache, and deploy in environments that primarily rely on native Transformers runtimes.

Limitations

• Full-quality mode is still the best default for important retrieval decisions; aggressive layer reduction trades away quality.
• Although the model supports up to 32K tokens, very long inputs still increase compute and memory cost.
• The model is optimized for retrieval and semantic similarity; some downstream tasks may benefit from task-specific fine-tuning.
• If your deployment stack requires ONNX out of the box, you will need to export that separately.

Citation

If you use this model, please cite the upstream work it is derived from:

@misc{mmbert-embed-2d-matryoshka,
  title={mmBERT-Embed: Multilingual Embedding Model with 2D Matryoshka Training},
  author={vLLM Semantic Router Team},
  year={2025},
  url={https://huggingface.co/llm-semantic-router/mmbert-embed-32k-2d-matryoshka}
}

License

Apache 2.0