CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

This is a Cross Encoder model finetuned from microsoft/MiniLM-L12-H384-uncased using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 256 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['what battery level should i leave my laptop at', 'So do yourself a favour and do it yourself. 1  Place a dead laptop battery in a completely sealed plastic bag. 2  Place the bag and battery in your freezer for between 11-14 hours. 3  Remove the bag from your freezer and allow the battery return to room temperature after taking it out of the bag, this is very important.'],
    ['what is gotu kola used for', "The remains of Kursk's reactor compartment was towed to Sayda Bay on Russia's northern Kola Peninsula – where more than 50 reactor compartments were afloat at pier points – after a shipyard had defueled the boat in early 2003. The rest of the boat was then dismantled for scrap."],
    ['does bcaa contain testosterone', 'Testosterone is a hormone produced by the testicles and is responsible for the proper development of male sexual characteristics. Testosterone is also important for maintaining muscle bulk, adequate levels of red blood cells, bone growth, a sense of well-being, and sexual function.s a man ages, the amount of testosterone in his body naturally gradually declines. This decline starts after age 30 and continues throughout life. Some causes of low testosterone levels are due to:'],
    ['what is the definition of the civil war word infantry', 'Captain America: Civil War Pre-order. To be notified when Captain America: Civil War becomes available on Blu-ray, simply submit your email below. You will now receive one email as soon as Captain America: Civil War becomes available.se the thumbs up and thumbs down icons to agree or disagree that the title is similar to Captain America: Civil War. You can also suggest completely new similar titles to Captain America: Civil War in the search box below.'],
    ['normal humans have how many autosomes', 'haploid(23) + haploid(23)=Diploid(46). Somatic cells in humans contain 46 chromosomes, and they are all diploid. Human gametes are haploid, and contain 23 chromosomesHuman cells have 23 pairs ( haploid) of large linear nuclear chromosomes, giving a total of 46 ( diploid ) per cell. ** Humans have special gonosomes ( sex chromosomes, in contrast to autosomes). These are xx in females and xy in males.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'what battery level should i leave my laptop at',
    [
        'So do yourself a favour and do it yourself. 1  Place a dead laptop battery in a completely sealed plastic bag. 2  Place the bag and battery in your freezer for between 11-14 hours. 3  Remove the bag from your freezer and allow the battery return to room temperature after taking it out of the bag, this is very important.',
        "The remains of Kursk's reactor compartment was towed to Sayda Bay on Russia's northern Kola Peninsula – where more than 50 reactor compartments were afloat at pier points – after a shipyard had defueled the boat in early 2003. The rest of the boat was then dismantled for scrap.",
        'Testosterone is a hormone produced by the testicles and is responsible for the proper development of male sexual characteristics. Testosterone is also important for maintaining muscle bulk, adequate levels of red blood cells, bone growth, a sense of well-being, and sexual function.s a man ages, the amount of testosterone in his body naturally gradually declines. This decline starts after age 30 and continues throughout life. Some causes of low testosterone levels are due to:',
        'Captain America: Civil War Pre-order. To be notified when Captain America: Civil War becomes available on Blu-ray, simply submit your email below. You will now receive one email as soon as Captain America: Civil War becomes available.se the thumbs up and thumbs down icons to agree or disagree that the title is similar to Captain America: Civil War. You can also suggest completely new similar titles to Captain America: Civil War in the search box below.',
        'haploid(23) + haploid(23)=Diploid(46). Somatic cells in humans contain 46 chromosomes, and they are all diploid. Human gametes are haploid, and contain 23 chromosomesHuman cells have 23 pairs ( haploid) of large linear nuclear chromosomes, giving a total of 46 ( diploid ) per cell. ** Humans have special gonosomes ( sex chromosomes, in contrast to autosomes). These are xx in females and xy in males.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

Unnamed Dataset

• Size: 79,518,560 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 10 charactersmean: 34.49 charactersmax: 115 characters	min: 70 charactersmean: 355.81 charactersmax: 920 characters	min: -11.81mean: 0.35max: 11.18

• Samples:

  sentence_0	sentence_1	label
  what battery level should i leave my laptop at	So do yourself a favour and do it yourself. 1 Place a dead laptop battery in a completely sealed plastic bag. 2 Place the bag and battery in your freezer for between 11-14 hours. 3 Remove the bag from your freezer and allow the battery return to room temperature after taking it out of the bag, this is very important.	-2.7446921666463218
  what is gotu kola used for	The remains of Kursk's reactor compartment was towed to Sayda Bay on Russia's northern Kola Peninsula – where more than 50 reactor compartments were afloat at pier points – after a shipyard had defueled the boat in early 2003. The rest of the boat was then dismantled for scrap.	-8.898806889851889
  does bcaa contain testosterone	Testosterone is a hormone produced by the testicles and is responsible for the proper development of male sexual characteristics. Testosterone is also important for maintaining muscle bulk, adequate levels of red blood cells, bone growth, a sense of well-being, and sexual function.s a man ages, the amount of testosterone in his body naturally gradually declines. This decline starts after age 30 and continues throughout life. Some causes of low testosterone levels are due to:	-6.447552680969238

• Loss: FitMixinLoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 1
• fp16: True

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs