SentenceTransformer based on intfloat/e5-base-v2

This is a sentence-transformers model finetuned from intfloat/e5-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: Does minor cross-contamination with Neste Renewable Diesel pose a risk to aviation jet fuel quality?',
    'passage: non-aromatic constituents exceeds aro- matic constituents”. Based on this, HVO belongs to the 2710… CN group of diesel fuels, although HVO’s origin is renewable. Logistics Neste Renewable Diesel and blends • behave like traditional fossil diesel fuel • no additional issues related to storage stability, water separation and microbiological growth • no additional material compatibility issues • small cross-contamination does not risk aviation jet fuel quality 26 There are Binding Tariff Information for HVO shown in',
    'passage: blends (20 % or less )** 2710 19 43 2710 19 43 30 Table 5. CN codes. Compatibility with materials Neste Renewable Diesel may be considered as having the same compatibility regarding parts and materials as conventional diesel, for example, with seals, hoses, diaphragms, dry couplers, and base swivel joints. Construction materials may include carbon and stainless steel which are suitable for conventionaldiesel fuel. The use of both welded and riveted tanks is acceptable. Tanks may have internal floating roofs',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.4956, 0.1537],
#         [0.4956, 1.0000, 0.3111],
#         [0.1537, 0.3111, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 667 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1
  type	string	string
  modality	text	text
  details	min: 16 tokensmean: 28.29 tokensmax: 40 tokens	min: 77 tokensmean: 113.63 tokensmax: 207 tokens

• Samples:

  sentence_0	sentence_1
  query: Why does the distillation range of Neste Renewable Diesel pose a lower risk of degraded lubrication compared to fossil fuels?	passage: • new emission control systems require even higher quality fuels • Neste Renewable Diesel is a high quality hydrocarbon enhancing quality of a fuel blend 37 Sulfur free fossil Neste Renewable Importance for vehicle technology diesel fuel Diesel in a blend and vehicle owner or as neat Distillation range Reasonable Better than in Lower engine oil dilution fossil fuels due to Lower risk for degraded lubrication lower FBP Long engine oil drain periods Cetane number Reasonable
  query: Why does the FQD refer to the EN 590 standard if it does not require diesel fuel to meet its overall specifications?	passage: of Justice has consistently held that Member States are not allowed to impose additional requirements on products the specifications of which have been fully harmonized at EU level. The RED and the FQD do not require diesel fuel to meet the EN 590 specifications. The FQD only refers to EN 590 in relation to the test methods in order to ensure that fuel producers and regulators use the same laboratory methods for demonstrating and monitoring compliance with its requirements (Article
  query: Under what classification is Neste Renewable Diesel categorized in Europe?	passage: Neste Renewable Diesel Handbook 1 Foreword This booklet provides information on Neste Renewable Diesel, in Europe classified as a Hydrotreated Vegetable Oil (HVO), and its use in diesel engines. Its potential readership consists of, e.g., fuel and exhaust emission professionals in oil companies, automotive industry representatives, fuel blenders, research facilities, and people preparing fuel standards and regulation. Contributors to this text have mainly been Ari Engman, Tuukka Hartikka, Markku Honkanen, Ulla Kiiski, Markku Kuronen, Kalle Lehto,

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• per_device_train_batch_size: 32
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 32
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Time

• Training: 20.7 seconds

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}