BGE base En v1.5 Phase 5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RishuD7/bge-base-en-v1.5-82-keys-phase-7-exp_v1")
# Run inference
sentences = [
    "(d) Names. Service Provider will not use the name of Company, any Affiliate of Company, any Company employee or any employee of any Affiliate of Company, or any product or service of Company or any of its Affiliates in any press release, advertising or materials distributed to prospective or existing customers, annual reports or any other public disclosure, except with Company's prior written authorization. Under no circumstances will Service Provider use the logos or other trademarks of Company or any of its Affiliates in any such materials or disclosures.\nService Provider Personnel shall, comply with any written instructions issued by Company with respect to.. the use, storage and handling of the Company Materials. Service Provider will use best efforts to protect the Company Materials from any loss of or damage while such Company Materials are under Service Provider's control, which control will be deemed to begin upon receipt of the Company Materials by. Service Provider; provided that Service Provider shall not be liable for any loss or damage to Company. Materials to the extent such loss or damage is caused by Service Provider's compliance with such written. instructions.",
    'Publicity',
    'CBRE_Termination Trigger - Client',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 8,031 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	anchor
  type	string	string
  details	min: 170 tokensmean: 377.79 tokensmax: 512 tokens	min: 3 tokensmean: 8.22 tokensmax: 11 tokens

• Samples:

  positive	anchor
  In the event that the Contractor provides the Service during the incomplete period of the the service was provided 3. The Customer shall reimburse the Contractor for the expenses incurred for the purchase of spare parts, equipment and materials for the purpose of providing the Services increased by the Contractor's mark-up, the amount of which is specified in Appendix No 4 "Terms and Conditions". The purchase of spare parts, equipment and materials referred to above will take place after the Contractor's application has been accepted by the Customer. 1 Settlement for the undertaking by the Contractor Emergency interventions will take place in accordance with and the conditions indicated in Clause 4 "Terms and Conditions" 5. For the performance of additional works, the Contractor will receive the remuneration specified in the application or contract for the performance of additional works accepted by the Client. Not later: within 7 (sleth) days from the date of termination of this Agre...	CBRE_Pricing Criteria
  4.1 The Contractor, despite a written warning issued by the Contractor by registered mail, violates the provisions of the Agreement and #$#cease the infringement within 14 (fourteen) days from the date of receipt of the summons from the Contractor, unless, . due to the nature of the breach. its removal. requires a longer period and the actions to remedy the breach are taken immediately. and duly by the Contractor:. 4.4 The Contractor shall cease to perform the duties resulting from the contract in part or in part for more than 3 days. 5. The Contractor may terminate the Contract with effect from the date of written service - under pain of non-wai:noscj - a statement of termination. if:. 5.1The Customer shall not comply with the obligation to submit the seals after the deadline for the payment of the two consecutive settlement periods specified on the invoice and after the deadline of fourteen days specified by the Contractor in the. written reminder; out business activity### S5 TER...	CBRE_Termination Trigger - Client
  Works commissioned to the Contractor, which do not fall within the scope of the contract, are additionally valued as Additional Works after prior acceptance of the Contractor's offer within 30 days from the date of delivery of the duly issued invoice issued after the. completion of these works. 7. The amount of remuneration due as set out in Schedule No 4 "Terms and Conditions shall be the net amount and shall beand VAT and VAT.. 8. Any discounts, commissions and other bonuses that the Contractor receivesin connection with its global purchasing program will be retained by the Contractor and I wil not be subiect to settlement with the Principal. 9. In the event of changes in the Iaw resulting in an increase in costs related to the provision of Services on the part of the Contractor, the Customer undertakes to cover the above- mentioned costs, documented by the Contractor.### S15 CONFIDENTIAL INFORMATION AND PROTECTION OF PERSONAL DATA 1.Any information obtained by the Customer or the C...	CBRE_WCP Status Criteria

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 30
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• tf32: False
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 30
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 3.3.1
• Transformers: 4.43.1
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}