SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: How does the reliance on cloud communication systems change with the increase in remote workforces due to the pandemic?',
    'The full extent of the impact of the COVID-19 pandemic on our business, operations and financial results will depend on numerous evolving factors that we may not be able to accurately predict, including those set forth under the section entitled "Risk Factors." In an effort to contain COVID-19 or slow its spread, governments around the world have enacted various measures, including orders to close non-essential businesses, isolate residents to their homes, and practice social distancing. To protect the health and safety of our employees, our workforce has spent significant time working from home and travel has been curtailed for our employees as well as our customers. Small and medium-sized customers have been particularly impacted by the COVID-19 pandemic. We have also experienced significant increases in usage by existing customers as our customers’ workforces are required to work from home in response to the COVID-19 pandemic accelerating trends we have seen in distributed workforces increasingly relying on cloud communication systems like ours. While we anticipate that the global health crisis caused by COVID-19 and the measures enacted to slow its spread will negatively impact business activity across the globe, it is not clear what its potential effects will be on our business, including the effects on our customers, suppliers or vendors, or on our financial results.\nCOMPONENTS OF RESULTS OF OPERATIONS\nService Revenue\nService revenue consists of communication services subscriptions, platform usage revenue, and related fees from our UCaaS, CCaaS, and CPaaS offerings. We plan to continue driving our business to increase service revenue through a combination of increased sales and marketing efforts, geographic expansion of our customer base outside the United States, innovation in product and technology, and through strategic acquisitions of technologies and businesses.\nOther Revenue\nOther revenue consists of revenues from professional services, primarily in support of deployment of our solutions and/or platform, and revenues from sales and rentals of IP telephones in conjunction with our cloud telephony service. Other revenue is dependent on the number of customers who choose to purchase or rent an IP telephone in conjunction with our service instead of using the solution on their cell phone, computer or other compatible device, and/or choose to engage our services for implementation and deployment of our cloud services.\nCost of Service Revenue',
    'We record deferred taxes based on differences between the financial statement basis and tax basis of assets and liabilities and available tax loss and credit carryforwards. In evaluating our ability to utilize our deferred tax assets, we consider available evidence, both positive and negative, in determining future taxable income on a jurisdiction-by-jurisdiction basis. We record a valuation allowance against deferred tax assets if, based on the weight of the evidence, it is more likely than not that some portion or all of the deferred tax assets will not be realized. A significant item of objective negative evidence considered was the historical three-year cumulative pretax loss reached in fiscal 2018. We continue to remain in a cumulative pretax loss position, and therefore, continued to maintain a full valuation allowance against our U.S., U.K., and Singapore deferred tax assets.\nLiquidity and Capital Resources\nAs of March 31, 2021, we had $152.9 million of cash and cash equivalents and short-term investments. In addition, we had $8.6 million in restricted cash in support of letters of credit securing leases for office facilities in California and New York. During fiscal 2021, $10.4 million previously held in escrow for our acquisition of Wavecell was released.\nAs of March 31, 2020, we had $170.9 million of cash and cash equivalents and short-term investments. In addition, we had $19.0 million in restricted cash, of which $8.6 million was in support of letters of credit securing leases for office facilities in California and New York and $10.4 million was held in escrow for our acquisition of Wavecell, pursuant to the terms of the acquisition agreement.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.5362, -0.0206],
#         [ 0.5362,  1.0000, -0.0907],
#         [-0.0206, -0.0907,  1.0000]])

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 6,857 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 20 tokensmean: 29.8 tokensmax: 57 tokens	min: 15 tokensmean: 403.25 tokensmax: 482 tokens

• Samples:

  sentence_0	sentence_1
  Represent this sentence for searching relevant passages: What was the electrical load estimated at for the mining operation as of December 31, 2020?	The Company owned approximately 669 and 649 Antminer S17 Pro Bitcoin miners located in LaFayette, GA as of December 31, 2020 and April 15, 2021, respectively. All miners were purchased from Bitmaintech Pte. Ltd., a Singapore limited company (“Bitmain”), and are collectively rated at approximately 30 Ph/s in computing power. Bitmain has acknowledged manufacturing defects, combined with inadequate repair facilities, rendering approximately one half of our miners in need of repair or replacement. The Company’s miners are housed in three modified shipping containers. The Company’s current electrical load is estimated at under 1.0 MW. The entire facility, including the land, two 2500 KVA 3-phase transformers, the mining containers, and miners, are owned by MGT. As the Company is presently using only a portion of the built-out available electrical load, it is exploring ways to grow and maintain its current operations including but not limited to further equipment sales, leasing space to othe...
  Represent this sentence for searching relevant passages: What is the accumulated deficit reported as of December 31, 2020, that raises doubt about the ability to continue operations?	The Company may raise additional capital, either through debt or equity financings, in order to achieve its business plan objectives. Management believes that it can be successful in obtaining additional capital; however, no assurance can be provided that the Company will be able to do so. There is no assurance, moreover, that any funds raised will be sufficient to enable the Company to attain profitable operations or continue as a going concern. To the extent that the Company is unsuccessful, the Company may need to curtail its operations and implement a plan to extend payables or reduce overhead until sufficient additional capital is raised to support further operations. The Company may also attempt to obtain funds through entering into arrangements with collaborative partners or others that may require the Company to relinquish rights to certain of our technologies or products that the Company would not otherwise relinquish. There can be no assurance that any such plan will be succe...
  Represent this sentence for searching relevant passages: What risks are associated with the development and acceptance of digital currencies that may impact operational results?	The Company may raise additional capital, either through debt or equity financings, in order to achieve its business plan objectives. Management believes that it can be successful in obtaining additional capital; however, no assurance can be provided that the Company will be able to do so. There is no assurance, moreover, that any funds raised will be sufficient to enable the Company to attain profitable operations or continue as a going concern. To the extent that the Company is unsuccessful, the Company may need to curtail its operations and implement a plan to extend payables or reduce overhead until sufficient additional capital is raised to support further operations. The Company may also attempt to obtain funds through entering into arrangements with collaborative partners or others that may require the Company to relinquish rights to certain of our technologies or products that the Company would not otherwise relinquish. There can be no assurance that any such plan will be succe...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 27.1 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}