SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RinKana/bge-base-en-v1.5-cross-lingual-retriever")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: What is the expected life expectancy in 2050?',
    "The 2013 New Zealand census enumerated a resident population of 4,242,048, an increase of 5.3% over the 2006 figure. As of June 2019, the total population has risen to an estimated 4,968,960. In 2018 the median age of the New Zealand population was 38.1 years.New Zealand is a predominantly urban country, with 73.0% of the population living in the seventeen main urban areas (i.e. population 30,000 or greater) and 55.1% living in the four largest cities of Auckland, Christchurch, Wellington, and Hamilton. New Zealand cities generally rank highly on international livability measures. For instance, in 2016 Auckland was ranked the world's third most liveable city and Wellington the twelfth by the Mercer Quality of Living Survey.Life expectancy for New Zealanders in 2012 was 84 years for females, and 80.2 years for males. Life expectancy at birth is forecast to increase from 80 years to 85 years in 2050 and infant mortality is expected to decline. New Zealand's fertility rate of 2.1 is relatively high for a developed country, and natural births account for a significant proportion of population growth. Consequently, the country has a young population compared to most industrialised nations, with 20% of New Zealanders being 14 years old or younger. By 2050 the median age is projected to rise from 36 years to 43 years and the percentage of people 60 years of age and older to rise from 18% to 29%. In 2008, the leading cause of premature death was cancer, at 29.8%, followed by ischaemic heart disease, 19.7%, and then cerebrovascular disease, 9.2%. As of 2016, total expenditure on health care (including private sector spending) is 9.2% of GDP.",
    "Niue is one of the world's largest coral islands. The terrain consists of steep limestone cliffs along the coast with a central plateau rising to about 60 metres (200 ft) above sea level. A coral reef surrounds the island, with the only major break in the reef being in the central western coast, close to the capital, Alofi. A notable feature is the number of limestone caves near the coast.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5540, 0.0874],
#         [0.5540, 1.0000, 0.2505],
#         [0.0874, 0.2505, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,379 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 14 tokensmean: 20.41 tokensmax: 40 tokens	min: 14 tokensmean: 225.98 tokensmax: 512 tokens

• Samples:

  sentence_0	sentence_1
  Represent this sentence for searching relevant passages: How else is Smithfield known?	Virginia's culture was popularized and spread across America and the South by figures such as George Washington, Thomas Jefferson, and Robert E. Lee. Their homes in Virginia represent the birthplace of America and the South. Modern Virginia culture has many sources, and is part of the culture of the Southern United States. The Smithsonian Institution divides Virginia into nine cultural regions.Besides the general cuisine of the Southern United States, Virginia maintains its own particular traditions. Virginia wine is made in many parts of the state. Smithfield ham, sometimes called "Virginia ham", is a type of country ham which is protected by state law, and can only be produced in the town of Smithfield. Virginia furniture and architecture are typical of American colonial architecture. Thomas Jefferson and many of the state's early leaders favored the Neoclassical architecture style, leading to its use for important state buildings. The Pennsylvania Dutch and their style can also be f...
  Represent this sentence for searching relevant passages: Who was the first to create the Man of Steel?	Ant-Man is the name of several fictional superheroes appearing in books published by Marvel Comics. Created by Stan Lee, Larry Lieber and Jack Kirby, Ant-Man's first appearance was in Tales to Astonish #35 (September 1962). The persona was originally the brilliant scientist Hank Pym's superhero alias after inventing a substance that can change size, but Scott Lang and Eric O'Grady also took on the mantle after the original changed his superhero identity to various other aliases, such as Giant-Man, Goliath, and Yellowjacket.
  Represent this sentence for searching relevant passages: Where's the white money?	Peacocks are a larger sized bird with a length from bill to tail of 100 to 115 cm (39 to 45 in) and to the end of a fully grown train as much as 195 to 225 cm (77 to 89 in) and weigh 4–6 kg (8.8–13.2 lb). The females, or peahens, are smaller at around 95 cm (37 in) in length and weigh 2.75–4 kg (6.1–8.8 lb). Indian peafowl are among the largest and heaviest representatives of the Phasianidae. So far as is known, only the wild turkey grows notably heavier. The green peafowl is slightly lighter in body mass despite the male having a longer train on average than the male of the Indian species. Their size, colour and shape of crest make them unmistakable within their native distribution range. The male is metallic blue on the crown, the feathers of the head being short and curled. The fan-shaped crest on the head is made of feathers with bare black shafts and tipped with bluish-green webbing. A white stripe above the eye and a crescent shaped white patch below the eye are formed by bare wh...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Time

• Training: 6.6 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}