SentenceTransformer based on intfloat/e5-base-v2

This is a sentence-transformers model finetuned from intfloat/e5-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: yakitori restaurant',
    'passage: - **Definition**: Non-commercial organizations serving community, membership, social-welfare, or civic purposes, including charities, associations, clubs, and community centers.\n        - **Key Concepts & Modalities**: Charitable nonprofits, membership associations, community centers, social service organizations, civic clubs, advocacy and mutual-aid groups.\n        - **Representative Keywords**: non profit organization, charity, community center, association, foundation, social services organization, youth organization, civic club, volunteer organization, NGO, membership organization, welfare organization, fraternal organization, community group, mutual aid.\n        - **Boundary Disambiguation**: Excludes government departments and places of worship whose primary identity is religious practice rather than secular community nonprofit activity. Community centers, civic centers, youth centers, veterans centers, senior citizen centers, and homeless shelters operated by nonprofit organizations belong here; those directly operated as government departments belong in Government & Public Services. The key distinction is nonprofit organizational structure versus government agency.',
    'passage: - **Definition**: Providers of essential utility infrastructure and energy services, including electricity, gas, water, waste, and related energy production or distribution facilities.\n        - **Key Concepts & Modalities**: Electric and gas utilities, water and wastewater services, waste management, energy generation, utility infrastructure services.\n        - **Representative Keywords**: electric utility, gas company, water utility, waste management service, power plant, solar energy company, wind farm, recycling center, sewage treatment, energy supplier, utility contractor, propane supplier, garbage collection, district heating, energy provider.\n        - **Boundary Disambiguation**: Excludes retail electronics or appliance stores and excludes general construction contractors who are not utility or energy operators. These are utility SERVICE providers delivering electricity, gas, or water directly to end customers. NOT factories or manufacturers. NOT construction contractors. The primary business is operating infrastructure and billing customers for consumption.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0123, -0.0051],
#         [-0.0123,  1.0000, -0.0121],
#         [-0.0051, -0.0121,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 48,425 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  modality	text	text
  details	min: 5 tokensmean: 6.93 tokensmax: 11 tokens	min: 144 tokensmean: 184.91 tokensmax: 248 tokens	min: 0.0mean: 0.07max: 1.0

• Samples:

  sentence_0	sentence_1	label
  query: concrete product supplier	passage: - Definition: Businesses that provide lodging, travel arrangement, tourism experiences, and visitor hospitality services for travelers. - Key Concepts & Modalities: Hotels and lodging, travel agencies, tour operators, guest accommodation, visitor hospitality services. - Representative Keywords: hotel, motel, hostel, bed and breakfast, travel agency, tour operator, resort hotel, guest house, vacation rental, tourist information center, lodge, inn, campground, booking agency, hospitality service, resort, rv park, airbnb. - Boundary Disambiguation: Excludes restaurants unless lodging is primary, and excludes passenger carriers such as airlines or bus companies that operate transport rather than tourism lodging or travel planning.	0.0
  query: photography class	passage: - Definition: Establishments that produce, fabricate, process, or assemble physical goods at industrial scale, including factories, plants, mills, and industrial production services. - Key Concepts & Modalities: Product manufacturing, industrial processing, fabrication and machining, plant and mill operations, industrial engineering support for production. - Representative Keywords: manufacturer, factory, industrial plant, machine shop, assembly plant, mill, foundry, chemical plant, electronics manufacturer, food manufacturer, textile mill, fabrication shop, production facility, cannery, printing plant. - Boundary Disambiguation: Excludes retail storefronts, wholesale-only distributors without production, and construction contractors who build on-site rather than manufacture products. Large-scale food manufacturers (canneries, frozen food manufacturers, food factories) belong here; breweries, distilleries, and wineries that serve or sell...	0.0
  query: museum of space history	passage: - Definition: Primary-sector operations involving farming, livestock, aquaculture, forestry, fishing, and extraction or stewardship of natural resources. - Key Concepts & Modalities: Crop and livestock farming, aquaculture and fishing, orchards and vineyards as production, agribusiness services, natural resource operations. - Representative Keywords: farm, ranch, orchard, vineyard, dairy farm, poultry farm, fishery, aquaculture farm, agricultural production, greenhouse, nursery grower, forestry service, livestock breeder, crop grower, agricultural cooperative. - Boundary Disambiguation: Excludes food restaurants and grocery retail, and excludes agricultural equipment manufacturers unless the site itself is a producing farm or resource operation. Vineyards, orchards, fish farms, and dairy farms as producers belong here; wineries that primarily serve wine to consumers on-premises belong in Food & Drink. Farm shops and farmstay businesses t...	0.0

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss",
      "cos_score_transformation": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• num_train_epochs: 6
• fp16: True
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• per_device_train_batch_size: 32
• num_train_epochs: 6
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 32
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Logs

Training Time

• Training: 1.1 hours

Framework Versions

• Python: 3.13.15
• Sentence Transformers: 5.7.0
• Transformers: 5.16.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.8.5
• Tokenizers: 0.23.1

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}