CrossEncoder based on jhu-clsp/ettin-encoder-17m

This is a Cross Encoder model finetuned from jhu-clsp/ettin-encoder-17m on the msmarco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: jhu-clsp/ettin-encoder-17m
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • msmarco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("tomaarsen/ms-marco-ettin-17m-reranker")
# Get scores for pairs of texts
pairs = [
    ['the name olmec means', 'The name Olmec comes from the Nahuatl word for the Olmecs: Å\x8clmÄ\x93catl [oË\x90lË\x88meË\x90kat͡ɬ] (singular) or Å\x8clmÄ\x93cah [oË\x90lË\x88meË\x90kaÊ\x94] (plural). This word is composed of the two words Å\x8dlli [Ë\x88oË\x90lË\x90i] , meaning rubber, and mÄ\x93catl [Ë\x88meË\x90kat͡ɬ] , meaning people, so the word means rubber people.hey lived in the tropical lowlands of south-central Mexico, in the present-day states of Veracruz and Tabasco. It has been speculated that Olmec derive in part from neighboring Mokaya and/or Mixeâ\x80\x93Zoque.'],
    ['what causes hissing in the ears', 'The following medical conditions are some of the possible causes of Hissing in ears. There are likely to be other possible causes, so ask your doctor about your symptoms. 1  Acute ear infection.  Chronic ear infection.'],
    ['multiple birth sibling definition', 'multiple birth(Noun) a birth in which more than one baby are born. Multiple birth. A multiple birth occurs when more than one fetus is carried to term in a single pregnancy. Different names for multiple births are used, depending on the number of offspring.'],
    ['what cause fluid build up on the knee', 'No: The joint should not have any appreciable fluid in it normally. Fluid can be there from many reasons. Arthritis can cause fluid. Also surgery can cause some bleeding in the knee which can cause the fluid to build up. The fluid makes the joint stiff and makes it hard to get a good quad function. ...Read more.'],
    ['how long is the wait list for kidney transplant', 'At the center of this is the simple fact that organ transplantation is built upon altruism and public trust. Gift of Life works hard to ensure that this trust is maintained, through its commitment both to the donor family as well as to those on the waiting list. Average Median Wait Time to Transplant. Kidney â\x80\x93 5 years.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'the name olmec means',
    [
        'The name Olmec comes from the Nahuatl word for the Olmecs: Å\x8clmÄ\x93catl [oË\x90lË\x88meË\x90kat͡ɬ] (singular) or Å\x8clmÄ\x93cah [oË\x90lË\x88meË\x90kaÊ\x94] (plural). This word is composed of the two words Å\x8dlli [Ë\x88oË\x90lË\x90i] , meaning rubber, and mÄ\x93catl [Ë\x88meË\x90kat͡ɬ] , meaning people, so the word means rubber people.hey lived in the tropical lowlands of south-central Mexico, in the present-day states of Veracruz and Tabasco. It has been speculated that Olmec derive in part from neighboring Mokaya and/or Mixeâ\x80\x93Zoque.',
        'The following medical conditions are some of the possible causes of Hissing in ears. There are likely to be other possible causes, so ask your doctor about your symptoms. 1  Acute ear infection.  Chronic ear infection.',
        'multiple birth(Noun) a birth in which more than one baby are born. Multiple birth. A multiple birth occurs when more than one fetus is carried to term in a single pregnancy. Different names for multiple births are used, depending on the number of offspring.',
        'No: The joint should not have any appreciable fluid in it normally. Fluid can be there from many reasons. Arthritis can cause fluid. Also surgery can cause some bleeding in the knee which can cause the fluid to build up. The fluid makes the joint stiff and makes it hard to get a good quad function. ...Read more.',
        'At the center of this is the simple fact that organ transplantation is built upon altruism and public trust. Gift of Life works hard to ensure that this trust is maintained, through its commitment both to the donor family as well as to those on the waiting list. Average Median Wait Time to Transplant. Kidney â\x80\x93 5 years.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Training Details

Training Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 39,770,704 training samples
• Columns: query_id, positive_id, negative_id, and score
• Approximate statistics based on the first 1000 samples:

  	query_id	positive_id	negative_id	score
  type	string	string	string	float
  details	min: 11 charactersmean: 33.5 charactersmax: 98 characters	min: 68 charactersmean: 358.26 charactersmax: 988 characters	min: 50 charactersmean: 346.46 charactersmax: 969 characters	min: -3.69mean: 13.4max: 22.15

• Samples:

  query_id	positive_id	negative_id	score
  how do you attach a sanding screen to a floor buffer	Lay the buffer on the floor so you can get underneath the housing and hook the screening brush to the central spindle. Center a sanding screen on the brush and push on it so it stays in place, then right the buffer and allow the housing to rest on the floor. Plug in the machine and grasp the handles with both hands before you turn it on.	The random orbital sander used an offset drive bearing that causes the pad to also move in an elliptical pattern. These pad movements are used in combination, randomly of course to help reduce the swirls that a non-random sheet sander might leave behind. Mechanics aside, a mojor difference is in the sanding medium. Most random orbital sanders use sanding disks, typically in a 5-inch diameter. They are attached to the sander's pad using hook & loop connections. The sanding disks have holes in them that match-up with the dust collection holes in the sander's pad.	15.936323801676433
  define method overriding	Method overriding is when a child class redefines the same method as a parent class, with the same parameters. For example, the standard Java class java.util.LinkedHashSet e…xtends java.util.HashSet. The method add() is overridden in LinkedHashSet.	In spite of its limitations, the MET concept provides a convenient method to describe the functional capacity or exercise tolerance of an individual as determined from progressive exercise testing and to define a repertoire of physical activities in which a person may participate safely, without exceeding a prescribed intensity level. PMID: 2204507	19.40437730153402
  toxic effects of constipation	by Jane Kriese. The colon is the part of the large intestine extending from the cecum to the rectum. And the most common sign of having a toxic colon is a condition called constipation. An accumulation of waste food byproducts in the colon can lead to constipation, toxic build up, weight gain and low energy.	In Summary. Commonly reported side effects of tizanidine include: bradycardia, dizziness, drowsiness, hypotension, asthenia, fatigue, and xerostomia. Other side effects include: constipation, and increased liver enzymes. See below for a comprehensive list of adverse effects.	7.669035658240318

• Loss: MarginMSELoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity"
  }
  

Evaluation Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 10,000 evaluation samples
• Columns: query_id, positive_id, negative_id, and score
• Approximate statistics based on the first 1000 samples:

  	query_id	positive_id	negative_id	score
  type	string	string	string	float
  details	min: 10 charactersmean: 34.37 charactersmax: 116 characters	min: 61 charactersmean: 341.63 charactersmax: 952 characters	min: 73 charactersmean: 346.27 charactersmax: 1350 characters	min: -0.85mean: 13.13max: 22.52

• Samples:

  query_id	positive_id	negative_id	score
  the name olmec means	The name Olmec comes from the Nahuatl word for the Olmecs: Ōlmēcatl [oːlˈmeːkat͡ɬ] (singular) or Ōlmēcah [oːlˈmeːkaʔ] (plural). This word is composed of the two words ōlli [ˈoːlːi] , meaning rubber, and mēcatl [ˈmeːkat͡ɬ] , meaning people, so the word means rubber people.hey lived in the tropical lowlands of south-central Mexico, in the present-day states of Veracruz and Tabasco. It has been speculated that Olmec derive in part from neighboring Mokaya and/or Mixe–Zoque.	Jai: Glory to, Joy, Hail. Guru; Gu - means darkness, and Ru - means a remover. So, Guru means a remover of darkness. Deva; means a shining one and is the source of the English word, Divine. So when we say Jai Guru Deva we are giving thanks to whoever we feel is the source of knowledge to us.	19.567786693572998
  what causes hissing in the ears	The following medical conditions are some of the possible causes of Hissing in ears. There are likely to be other possible causes, so ask your doctor about your symptoms. 1 Acute ear infection. Chronic ear infection.	There are many possible causes of itchy ears. Although the annoyance of itchy ears is not usually found to be serious, the condition is very real no matter the reason for the uncomfortable nuisance. Poking objects like cotton swabs, pencils, and paperclips in the ears to relieve the irritation is a dangerous and non-effective method for relief.	10.332231203715006
  multiple birth sibling definition	multiple birth(Noun) a birth in which more than one baby are born. Multiple birth. A multiple birth occurs when more than one fetus is carried to term in a single pregnancy. Different names for multiple births are used, depending on the number of offspring.	From Wikipedia, the free encyclopedia. Not to be confused with Life unworthy of life or Wrongful birth. Wrongful life is the name given to a legal action in which someone is sued by a severely disabled child (through the child's legal guardian) for failing to prevent the child's birth.1 1 Definition.rongful life is the name given to a legal action in which someone is sued by a severely disabled child (through the child's legal guardian) for failing to prevent the child's birth. Contents.	16.912671327590942

• Loss: MarginMSELoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• load_best_model_at_end: True

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

• The bold row denotes the saved checkpoint.

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Energy Consumed: 4.348 kWh
• Carbon Emitted: 1.605 kg of CO2
• Hours Used: 1.576 hours

Training Hardware

• On Cloud: No
• GPU Model: 8 x NVIDIA H100 80GB HBM3
• CPU Model: AMD EPYC 7R13 Processor
• RAM Size: 1999.99 GB

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.9.1+cu126
• Accelerate: 1.12.0
• Datasets: 4.4.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}