Contextual AI Reranker v2 1B — SequenceClassification (single-logit) Converted Model

This repository contains a drop-in SequenceClassification version of the original ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b. It exposes a single logit per input (one score) that is numerically equivalent to the original model’s last-token vocab_id=0 logit (next_logits[:, 0]). That means you can use standard text-classification/CrossEncoder tooling for fast, simple reranking—without custom logits processors—while preserving the original scores and ranking order.

What changed? We copy the LM head’s row 0 vector into a 1-logit classification head (score.weight ← lm_head.weight[0]), set bias to 0 (or the matching bias row if present), and keep tokenizer/padding behavior aligned with the original. Result: SequenceClassification output ≡ original next_logits[:, 0].

Highlights

• Parity with the original: The score from this model equals the original ID=0 logit at the very last token position (use the same prompt template and left-padding).
• Frictionless integration: Works out-of-the-box with Sentence-Transformers CrossEncoder and standard Transformers classification interfaces.
• Fast & memory-light: Computes a single logit (hidden_size × 1) instead of a full vocabulary projection.
• Multilingual and long-context (inherits capabilities from the base reranker).

Model Overview

• Type: Text Reranking (single-logit SequenceClassification)
• Base: ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b (Qwen3 CausalLM)
• Languages: 100+ (inherited)
• Params: ~1B (inherited)
• Context Length: up to 32K (inherited)
• Scoring definition: single logit ≡ original next_logits[:, 0]

Input Formatting (keep this template)

Check whether a given document contains information helpful to answer the query.
<Document> {document}
<Query> {query}{optional_instruction} ??

• Use left padding so the last token aligns across a batch.
• If the tokenizer has no pad_token, set pad_token = eos_token.

Updated Usage

Below are drop-in examples for the converted model. These mirror the original card’s behavior but through SequenceClassification.

Updated Sentence Transformers Usage (CrossEncoder)

from sentence_transformers import CrossEncoder

MODEL_ID = "sigridjineth/ctxl-rerank-v2-1b-seq-cls"  # or local folder

def format_prompts(query: str, instruction: str, docs: list[str]) -> list[str]:
    inst = f" {instruction}" if instruction else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {query}{inst} ??"
        for d in docs
    ]

query = "Which is a domestic animal?"
docs = ["Cats are pets.", "The moon is made of cheese.", "Dogs are loyal companions."]

ce = CrossEncoder(MODEL_ID, max_length=8192)

# Ensure original padding behavior
if ce.tokenizer.pad_token is None:
    ce.tokenizer.pad_token = ce.tokenizer.eos_token
ce.tokenizer.padding_side = "left"

prompts = format_prompts(query, "", docs)
scores = ce.predict(prompts)  # one logit per doc (higher = more relevant)

ranked = sorted(zip(scores, docs), key=lambda x: x[0], reverse=True)
for s, d in ranked:
    print(f"{s:.4f} | {d}")

Updated Transformers Usage (SequenceClassification)

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_ID = "sigridjineth/ctxl-rerank-v2-1b-seq-cls"  # or local folder
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

def format_prompts(query: str, instruction: str, docs: list[str]) -> list[str]:
    inst = f" {instruction}" if instruction else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {query}{inst} ??"
        for d in docs
    ]

tok = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
tok.padding_side = "left"

model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, torch_dtype=dtype).to(device).eval()

query = "Which is a domestic animal?"
docs = ["Cats are pets.", "The moon is made of cheese."]
prompts = format_prompts(query, "", docs)

enc = tok(prompts, return_tensors="pt", padding=True, truncation=True).to(device)
with torch.no_grad():
    logits = model(**enc).logits.squeeze(-1)     # [batch]
    # Optional: exact parity rounding with original BF16 readout
    scores = logits.to(torch.bfloat16).float().cpu().tolist()

ranked = sorted(zip(scores, docs), key=lambda x: x[0], reverse=True)
for s, d in ranked:
    print(f"{s:.4f} | {d}")

Note on parity: Casting the output logit to bf16 then back to float matches the original card’s BF16 rounding step.

(Reference) Original Transformers Usage (CausalLM)

If you prefer to call the original model directly, compute next_logits[:, -1, 0] as specified in the base card.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

BASE_ID = "ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

def format_prompts(q: str, inst: str, docs: list[str]) -> list[str]:
    inst = f" {inst}" if inst else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {q}{inst} ??"
        for d in docs
    ]

tok = AutoTokenizer.from_pretrained(BASE_ID, use_fast=True)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
tok.padding_side = "left"

lm = AutoModelForCausalLM.from_pretrained(BASE_ID, torch_dtype=dtype).to(device).eval()

docs = ["Cats are pets.", "The moon is made of cheese."]
prompts = format_prompts("Which is a domestic animal?", "", docs)
enc = tok(prompts, return_tensors="pt", padding=True, truncation=True).to(device)

with torch.no_grad():
    out = lm(**enc).logits[:, -1, :]        # [batch, vocab]
    scores = out[:, 0].to(torch.bfloat16).float().cpu().tolist()

for s, d in sorted(zip(scores, docs), key=lambda x: x[0], reverse=True):
    print(f"{s:.4f} | {d}")

Conversion Details

• Architecture: Qwen3ForSequenceClassification(num_labels=1)

• Head initialization:

  • score.weight ← lm_head.weight[0] (row for vocab_id=0)
  • score.bias ← 0 (or the corresponding bias term if present in LM head)

• Tokenizer/Config:

  • Ensure pad_token exists (pad_token = eos_token if missing)
  • Set padding_side="left"
  • Propagate pad/eos/bos IDs into the model config for correct batching

• Parity check:

  • Verified that SequenceClassification logit ≡ original next_logits[:, 0]
  • Optional BF16 round-trip on the score for exact rounding parity

Intended Use & Limitations

• Use: Document reranking for search/QA/multilingual scenarios; batch scoring of (query, document) prompts.
• Not for: Open-ended generation; the model emits a single score per input.
• License constraints: Non-commercial & Share-Alike. If you redistribute derivatives, include attribution and the same license.
• Bias & safety: Inherits all limitations and potential biases of the base model; evaluate before deployment.

Requirements

• Transformers ≥ 4.51.0
• PyTorch with BF16 support recommended on GPU
• Long inputs: set max_length accordingly (up to the inherited context window)

Citation

If you use this converted model, please cite the original work:

@misc{ctxl_rerank_v2_instruct_multilingual,
  title = {Contextual AI Reranker v2},
  author = {George Halal and Sheshansh Agrawal and Bo Han and Arnav Palkhiwala},
  year = {2025},
  url = {https://contextual.ai/blog/rerank-v2}
}

License

This repository follows the original Creative Commons Attribution Non Commercial Share Alike 4.0 (CC-BY-NC-SA-4.0) license. You must provide attribution, may not use it commercially, and must distribute derivatives under the same license.

Acknowledgements

All modeling, training, and evaluation credit goes to Contextual AI for the original ctxl-rerank-v2 family. This repository provides a compatibility conversion to a single-logit SequenceClassification interface for easier integration and deployment.