all-MiniLM-L6-v24-pair_score

This is a sentence-transformers model finetuned from KhaledReda/all-MiniLM-L6-v23-pair_score on the pairs_with_scores_v120_tag_true_positives_and_false_negatives_description dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: KhaledReda/all-MiniLM-L6-v23-pair_score
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pairs_with_scores_v120_tag_true_positives_and_false_negatives_description
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'laces boot',
    'nursing covers mustard flowers category kids baby care breastfeeding aid breastfeeding aid tags breathable nursing cover full coverage nursing cover foldable nursing cover pouch nursing cover colorful nursing cover flowers nursing covers mustard nursing covers nursing covers keywords flowers nursing covers mustard nursing covers nursing covers description breastfeeding is one of the most special yet challenging things in motherhood we just wanted to add some more colors to this special moment with all its colors product details soft light breathable fabric machine washable full coverage comes with its pouch foldable in seconds',
    'raw african coffee soap category beauty skincare face soap face soap tags shea butter soap coconut oil soap antioxidant soap firming skin soap dark spots soap coffee soap raw african raw african soap soap ahwa soap kahwa soap kahwah soap qahwa soap raw african ahwa soap raw african kahwa soap raw african kahwah soap raw african qahwa soap keywords coffee soap raw african raw african soap soap ahwa soap kahwa soap kahwah soap qahwa soap raw african ahwa soap raw african kahwa soap raw african kahwah soap raw african qahwa soap description our coffee-based soap bar gives you a boosting and energizing sensation this soap is rich in antioxidants and nutrients that fight age signs firms and tighten the skin and gives you a youthful look. it helps in reducing dark spots and acne scars. for all skin types. this product is free of harsh chemicals like parabens sulphates or mineral oils. we never test our products on animals and we don t deal with suppliers who test their products on animals. ingredients shea butter coconut oil olive oil coffee.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0817, -0.0683],
#         [-0.0817,  1.0000, -0.0380],
#         [-0.0683, -0.0380,  1.0000]])

Training Details

Training Dataset

pairs_with_scores_v120_tag_true_positives_and_false_negatives_description

• Dataset: pairs_with_scores_v120_tag_true_positives_and_false_negatives_description at 25785dc
• Size: 12,582,766 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 3 tokensmean: 5.59 tokensmax: 22 tokens	min: 11 tokensmean: 104.43 tokensmax: 256 tokens	min: 0.0mean: 0.1max: 1.0

• Samples:

  sentence1	sentence2	score
  bergamot eau de toilette	knitted crop top in beige category fashion casual wear top top tags women top beige top comfort top breathable top fabric top stretch top knitted sets crop top top keywords crop top top attrs gender women brand psych generic name top size s features high-waisted soft breathable cut cropped material knitted color beige occasion beach description elevate your style with our chic knitted set featuring a matching pair of knitted pants and a knitted crop top. designed for comfort and effortless elegance this set is perfect for any occasion. the knitted crop top offers a flattering fit with a touch of stretch while the high-waisted knitted pants provide a sleek silhouette and ultimate comfort. made from soft breathable fabric this set is perfect for both enjoying a day at the beach and stepping out in style. versatile and stylish this knitted set is a must-have addition to your wardrobe. mix and match with your favorite accessories for a look that s uniquely you. model is 177 cm wearing size...	0.0
  wide leg pants	titania solingen no/1063 category beauty cosmetics make-up tool tweezers tags solingen tweezers titania titania tweezers tweezers keywords solingen tweezers titania titania tweezers tweezers	0.0
  women pumps	neurimax 30/cap 2 ex.new category health and nutrition dietary supplements joint supplement joint supplement tags neurimax neurimax supplement keywords neurimax neurimax supplement attrs pharmacies form cap	0.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

pairs_with_scores_v120_tag_true_positives_and_false_negatives_description

• Dataset: pairs_with_scores_v120_tag_true_positives_and_false_negatives_description at 25785dc
• Size: 63,230 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 3 tokensmean: 5.51 tokensmax: 20 tokens	min: 13 tokensmean: 104.12 tokensmax: 256 tokens	min: 0.0mean: 0.1max: 1.0

• Samples:

  sentence1	sentence2	score
  always maxi long	american - buffalo chicken hot category restaurants pizza deli pizza tags cheesy american pizza thick pizza mozzarella pizza cheese pizza hot chicken pizza american pizza buffalo chicken pizza chicken pizza pizza keywords american pizza buffalo chicken pizza chicken pizza pizza description a thick pizza with a generous amount of cheese mozzarella tomato sauce chicken buffalo sauce	0.0
  sushi	lemon fluffy set category fashion casual wear outfit outfit tags linen outfit summer outfit women outfit blouse skirt fluffy outfit lemon outfit outfit outfit set keywords fluffy outfit lemon outfit outfit outfit set attrs gender women brand dovera generic name outfit size one size features fluffy outfit style skirt top material linen color lemon season summer description modest comfyand summery set with fluffy skirt and top comes in 3 colors apple green - brown - blue one size. outside materials linen. blouse length 55 cm width 65 cm shoulder 25 cm skirt length 100 cm	0.0
  eyefree lid wipes	disposable - 5 ml syringe - latex free - 1 pcs category pharmacies first aid and medical equipment medical accessory medical accessory tags disposable syringe syringe keywords disposable syringe syringe attrs units 1 pcs 5 millilitre	0.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.1.0
• Transformers: 4.55.4
• PyTorch: 2.6.0+cu124
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}