Iso-ModernColBERT

This model is an isotropically corrected version of GTE-ModernColBERT-v1. It's built for production use cases where retrieval speed and quality matter. Compared to the original model, this version delivers up to 3x faster inference in bf16 with almost no loss in accuracy and enables scalable multi-vector retrieval through Sparse Multi-Vector Encoding (SMVE) inside TopK.

Usage

Sentence Transformers

This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:

pip install "sentence-transformers>=6.0.0"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("topk-io/Iso-ModernColBERT")

query = "Which planet is known as the Red Planet?"
documents = [
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
]

query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# (12, 128) (18, 128)

# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[ 9.4844, 10.4180,  9.8516, 10.1953]])

PyLate

Install PyLate for embeddings and TopK SDK for retrieval.

pip install -U pylate topk-sdk

Embed documents

First, load the model into PyLate ColBERT class and encode your documents.

import torch
import numpy as np
from pylate import models


model = models.ColBERT(
  model_name_or_path="topk-io/Iso-ModernColBERT",
  model_kwargs={'torch_dtype': torch.bfloat16},
)

documents = [
  "document 1 text",
  "document 2 text",
  "document 3 text",
]

doc_embeddings = model.encode(
    documents,
    batch_size=32,
    # Ensure that it is set to False to indicate that these are documents, not queries
    is_query=False,
    show_progress_bar=True,
)

Store document embeddings

Index multi-vector document embeddings inside TopK, hybrid retrieval engine built on object storage. To get started, create an API key.

from topk_sdk import Client
from topk_sdk.schema import matrix, multi_vector_index

# Initialize TopK client
client = Client(
    api_key = "<TOPK_API_KEY>",
    region = "aws-us-east-1-elastica",
)

# Create a collection with multi-vector index
client.collections().create(
    "iso-moderncolbert",
    schema = {
        "token_embeddings": matrix(dimension=128, value_type="f16")
            .index(multi_vector_index(metric="maxsim"))
    }
)

# Upsert document embeddings
client.collection("iso-moderncolbert").upsert([
  {
    "_id": str(i),
    "token_embeddings": emb.astype(np.float16),
    "text": text
  }
  for (i, (text, emb)) in enumerate(zip(documents, doc_embeddings))
])

Retrieve documents for queries

Your documents are now durably persisted in the index and queryable.

from topk_sdk.query import fn, select, field

# Encode query string
query_embedding = model.encode(
    "query for document 3",
    # Ensure that it is set to True for queries
    is_query=True,
    show_progress_bar=False,
)

# Retrieve top-k documents using the query embedding
results = client.collection("iso-moderncolbert").query(
    select(
        "_id", "text",
        # Compute maxsim between query and indexed documents
        maxsim_score = fn.multi_vector_distance(
            "token_embeddings",
            query_embedding.astype(np.float16)
        )
    )
    # Get the top 10 matching documents
    .topk(field("maxsim_score"), 10)
)

for r in results:
    print(f"id: {r['_id']}, score: {r['maxsim_score']}, text: {r['text']}")

TopK's query language is flexible and allows you to tune retrieval parameters, combine multi-vector with metadata filters, keyword search, and more. Check out our docs to learn more.

Evaluation results

We conducted evaluation of our model using an internal evaluation harness on two standard benchmarks - BEIR and NanoBEIR. For baselines, we selected GTE-ModernColBERT-v1 and evaluated its perfomance in fp32 and bf16 precision (denoted by GTE fp32 and GTE bf16, respectively). The last two columns of each table — Iso bf16 and Δ vs GTE — describe Iso-ModernColBERT (ours) in bf16 precision. In all configurations we used the same SMVE implementation with width 65536 and k=32.

BEIR

NDCG@10 — ranking quality is robust to bf16

End-to-end ranking quality reported as NDCG@10, using exact MaxSim scoring (no approximation). GTE-ModernColBERT-v1 loses ~7 NDCG points on average going from fp32 → bf16 — about a 13% relative drop — with the worst-hit datasets (trec-covid, climate-fever, hotpotqa) dropping 12–16 points. Iso-ModernColBERT keeps fp32-level ranking quality in bf16, recovering most of that gap on average and on every dataset.

Recall@100 — SMVE as a first stage with ~10× overfetch

The following results show model performance when used with Sparse Multi-Vector Encoder (SMVE) as a first stage retriever.

For a SMVE first stage to be usable, it needs to surface the candidates that the exact fp32 MaxSim model would have ranked at the top. SMVE on GTE-ModernColBERT-v1 is broken — its compacted latent geometry means random anchors don't separate vectors well. Iso-ModernColBERT's SMVE recovers (and often exceeds) the fp32 MaxSim top-10 within 10× overfetch.

⚠ The +11,812% on climate-fever is an artifact of a near-zero baseline (0.41%): GTE's SMVE is so broken on that dataset that the ratio explodes. Read it as "GTE SMVE doesn't work here at all", not as a meaningful magnitude.

Recall@1000 — SMVE as a first stage with ~10× overfetch (deeper pool)

Same picture at the next pool depth: Iso-ModernColBERT SMVE R@1000 essentially matches or exceeds fp32 MaxSim R@100 across the board, while GTE's SMVE collapses.

⚠ Again, climate-fever's +7,305% is driven by a near-zero baseline (0.93%) — GTE SMVE simply doesn't work on this dataset.

NanoBEIR

NDCG@10 — ranking quality is robust to bf16

End-to-end ranking quality reported as NDCG@10, using exact MaxSim scoring (no approximation). GTE-ModernColBERT-v1 drops ~6 NDCG points on average going from fp32 → bf16 — about a 9% relative drop — with some datasets (ArguAna, ClimateFEVER, FiQA, Touche2020) losing 8–13 points. Iso-ModernColBERT keeps fp32-level ranking quality in bf16 — average is within 0.6 points of fp32, and most per-dataset gaps close to a few percent.

Recall@100 — SMVE as a first stage with ~10× overfetch

The following results show model performance when used with Sparse Multi-Vector Encoder (SMVE) as a first stage retriever.

For a SMVE first stage to be usable, it needs to surface the candidates that the exact fp32 MaxSim model would have ranked at the top. SMVE on GTE-ModernColBERT-v1 is broken — its compacted latent geometry means random anchors don't separate vectors well. Iso-ModernColBERT's SMVE recovers (and often exceeds) fp32 MaxSim's top-10 within 10× overfetch.

Recall@1000 — SMVE as a first stage with ~10× overfetch (deeper pool)

Same picture at the next pool depth: Iso-ModernColBERT SMVE R@1000 essentially matches or exceeds fp32 MaxSim R@100 across the board, while GTE's SMVE consistently undershoots.