Redis fine-tuned CrossEncoder model for semantic caching on LangCache

This is a Cross Encoder model finetuned from Alibaba-NLP/gte-reranker-modernbert-base on the LangCache Sentence Pairs (subsets=['all'], train+val=True) dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for sentence pair classification.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: Alibaba-NLP/gte-reranker-modernbert-base
• Maximum Sequence Length: 8192 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • LangCache Sentence Pairs (subsets=['all'], train+val=True)
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("redis/langcache-reranker-v1-softmnrl-triplet")
# Get scores for pairs of texts
pairs = [
    [' What high potential jobs are there other than computer science?', ' What high potential jobs are there other than computer science?'],
    [' Would India ever be able to develop a missile system like S300 or S400 missile?', ' Would India ever be able to develop a missile system like S300 or S400 missile?'],
    [' water from the faucet is being drunk by a yellow dog', 'A yellow dog is drinking water from the faucet'],
    [' water from the faucet is being drunk by a yellow dog', 'The yellow dog is drinking water from a bottle'],
    ['! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``', '! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    ' What high potential jobs are there other than computer science?',
    [
        ' What high potential jobs are there other than computer science?',
        ' Would India ever be able to develop a missile system like S300 or S400 missile?',
        'A yellow dog is drinking water from the faucet',
        'The yellow dog is drinking water from a bottle',
        '! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

LangCache Sentence Pairs (subsets=['all'], train+val=True)

• Dataset: LangCache Sentence Pairs (subsets=['all'], train+val=True)
• Size: 1,452,533 training samples
• Columns: anchor, positive, and negative_1
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative_1
  type	string	string	string
  details	min: 24 charactersmean: 114.25 charactersmax: 268 characters	min: 19 charactersmean: 114.1 charactersmax: 226 characters	min: 4 charactersmean: 93.04 charactersmax: 234 characters

• Samples:

  anchor	positive	negative_1
  Any Canadian teachers (B.Ed. holders) teaching in U.S. schools?	Any Canadian teachers (B.Ed. holders) teaching in U.S. schools?	Are there many Canadians living and working illegally in the United States?
  Are there any underlying psychological tricks/tactics that are used when designing the lines for rides at amusement parks?	Are there any underlying psychological tricks/tactics that are used when designing the lines for rides at amusement parks?	Is there any tricks for straight lines mcqs?
  Can I pay with a debit card on PayPal?	Can I pay with a debit card on PayPal?	Can you transfer PayPal funds onto a debit card/credit card?

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "num_negatives": 1,
      "activation_fn": "torch.nn.modules.activation.Sigmoid"
  }
  

Evaluation Dataset

LangCache Sentence Pairs (split=test)

• Dataset: LangCache Sentence Pairs (split=test)
• Size: 110,066 evaluation samples
• Columns: anchor, positive, and negative_1
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative_1
  type	string	string	string
  details	min: 3 charactersmean: 97.95 charactersmax: 314 characters	min: 3 charactersmean: 97.03 charactersmax: 314 characters	min: 11 charactersmean: 74.49 charactersmax: 295 characters

• Samples:

  anchor	positive	negative_1
  What high potential jobs are there other than computer science?	What high potential jobs are there other than computer science?	Why IT or Computer Science jobs are being over rated than other Engineering jobs?
  Would India ever be able to develop a missile system like S300 or S400 missile?	Would India ever be able to develop a missile system like S300 or S400 missile?	Should India buy the Russian S400 air defence missile system?
  water from the faucet is being drunk by a yellow dog	A yellow dog is drinking water from the faucet	Do you get more homework in 9th grade than 8th?

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "num_negatives": 1,
      "activation_fn": "torch.nn.modules.activation.Sigmoid"
  }
  

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.1.0
• Transformers: 4.56.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}