SentenceTransformer based on intfloat/e5-large

This is a sentence-transformers model finetuned from intfloat/e5-large. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-large
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'query: factory reset in DSU mode without permission check?',
]
documents = [
    'passage: In rebootWipeUserData of RecoverySystem.java, there is a possible way to factory reset the device while in DSU mode due to a missing permission check. This could lead to physical denial of service with no additional execution privileges needed. User interaction is not needed for exploitation.',
    'passage: A cross-site scripting (XSS) vulnerability in /index.php/?p=report of Online Fire Reporting System v1.0 allows attackers to execute arbitrary web scripts or HTML via a crafted payload injected into the "Contac #" text field.',
    'passage: Multiple stack-based buffer overflows in Mini-stream Easy RM-MP3 Converter 3.0.0.7 allow remote attackers to execute arbitrary code via (1) a long rtsp URL in a .ram file and (2) a long string in the HREF attribute of a REF element in a .asx file.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.6806, -0.1250, -0.1078]])

Evaluation

Metrics

Information Retrieval

• Dataset: validation
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 861,607 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1
  type	string	string
  modality	text	text
  details	min: 9 tokensmean: 14.16 tokensmax: 32 tokens	min: 22 tokensmean: 94.61 tokensmax: 512 tokens

• Samples:

  sentence_0	sentence_1
  query: why are my data records disappearing?	passage: ---title: Cross-Tenant Legacy Correlation Disclosure and Deletiondraft: falsehero: image: /static/img/heros/hero-legal2.svg content: "# Cross-Tenant Legacy Correlation Disclosure and Deletion"date: 2026-01-29product: Grafanaseverity: Lowcve: CVE-2026-21727cvss_score: "3.3"cvss_vector: "CVSS:3.3/AV:N/AC:H/PR:H/UI:N/S:U/C:L/I:L/A:N"fixed_versions: - ">=11.6.11 >=12.0.9 >=12.1.6 >=12.2.4"---A cross-tenant isolation vulnerability was found in Grafana’s Correlations feature affecting legacy correlation records. Due to a backward compatibility condition allowing org_id = 0 records to be returned across organizations, a user with datasource management privileges could read and permanently delete legacy correlation data belonging to another organization. This issue affects correlations created prior to Grafana 10.2 and is fixed in >=11.6.11, >=12.0.9, >=12.1.6, and >=12.2.4.Thanks to Gyu-hyeok Lee (g2h) for reporting this vulnerability.
  query: Exim 4.7x DKIM identity issue	passage: The DKIM implementation in Exim 4.7x before 4.76 permits matching for DKIM identities to apply to lookup items, instead of only strings, which allows remote attackers to execute arbitrary code or access a filesystem via a crafted identity.
  query: JMK Picture Gallery security issue	passage: JMK's Picture Gallery allows remote attackers to bypass authentication via a direct request to admin_gallery.php3, possibly related to the add action.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• num_train_epochs: 1
• fp16: True
• per_device_eval_batch_size: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• per_device_train_batch_size: 4
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 4
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 1.2 days
• Evaluation: 18.2 minutes
• Total: 1.2 days

Framework Versions

• Python: 3.12.10
• Sentence Transformers: 5.5.1
• Transformers: 5.9.0
• PyTorch: 2.11.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}