SentenceTransformer based on intfloat/multilingual-e5-base

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: March 17, 2019-a ‘Yeshu Sabka Sahara’ tih thupui hmanga thusawi tuin nge sawi?',
    'passage: Lianhmingthanga Sailo, Senior Executive Secretary leh Upa R. Lallianzuala te an kal. March 13 - 18, 2019 chhung hian KAPC Synod hmanpuiin Rev. B. Sangthanga, Executive Secretary leh Upa Lalhmingliana te an kal. Hruaina Eng FM Radio MARCH 15, 2019 (FM Zoawi Aizawl & FM Lunglei @ 7:30 p.m.): Zaipawl:Besy Choir - Mawhphurhna kan ne, Solo: RTC Lalduhawmi - Kan ngaithla che, Thusawi: Upa DP Biakkhuma - Chanchin ha hril tihmakmawh, Solo: Gracy Vl Pachuau - Lalpa chu chhuang rawh se, Zaipawl: TBZ Choir - ṭanpuitu an ngai em e Hruaina Eng TV Programme - Susamachar Prasaran Programme MARCH 17, 2019 (Pathianni) 5:00 pm. : Group/Choir/Solo : Siyon Swaram - Tere Charno, Group/Choir/Solo : Sihphir Bial - Me Akhen Apni, Sermon : Zohmingthanga - Yeshu Sabka Sahara, Group/Choir/Solo : Bungkawn Vengthar -Siyon Desh, Group/Choir/Solo : Venghnuai Kohhran - Ham Syon Ka MARCH 17, 2019 (Tv Programme) @ 4:00 - 6:00, 8:00 - 9:00 p.m) : Weekly Round Up, KTP Rorel Inkhawm 2019, Kohhran Hmeichhe Inkhawmpui Lian 2019 Synod Social Front Helpline: 9089777129; Tuai & Patil Helpline Students’ Helpline : Synod Information & Publicity Phone - 2314648 Synfo : 8729800130; 9856883377',
    'passage: : Group/Choir/Solo : Mission Veng Masihi Sangati - Jisu Daken, Group/Choir/Solo : Siyon Swaram - Tere Charno, Sermon : Zohmingthanga - Yeshu Sabka Sahara, Group/Choir/Solo : Venghnuai Kohhran -AwAkash, Group/Choir/Solo : Khatla Masihi Sangati - Ishu Tera Wada Moderator Programme Synod Moderator Rev.Lalrinmawia chuan tunkar tawpah hetiang hian hun a hmang ang : Laipuitlang Kohhran, Chanmari Bial October 17, 2015 (Inrinni) Zan Chawhma Mualkhang Kohhran, Sairang Dinthar Bial October 18, 2015 (Pathianni) Chawhnu Khamrang Kohhran, Sairang Dinthar Bial October 18, 2015 (Pathianni) Dinthar Kohhran, Sairang Dinthar Bial October 18, 2015 (Pathianni) Zan Synod Social Front Helpline: 9089777129; Tuai & Patil Helpline Students’ Helpline : Synod Information & Publicity Phone - 2314648 Synfo : 8729800130; 9856883377',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5381, 0.2664],
#         [0.5381, 1.0000, 0.2270],
#         [0.2664, 0.2270, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 82,630 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative
  type	string	string	string
  modality	text	text	text
  details	min: 15 tokensmean: 27.64 tokensmax: 40 tokens	min: 90 tokensmean: 322.55 tokensmax: 512 tokens	min: 111 tokensmean: 361.2 tokensmax: 512 tokens

• Samples:

  anchor	positive	negative
  query: Jamal Khashoggi chungchangah hian eng news organization nge Mohammed bin Salman interview?	passage: "Heng zawng zawng hi a tawp tur angin han ngaihtuah chhin teh u. Chumi awmzia chu khawvel economy a tluchhe dawn tihna a ni a, Saudi Arabia emaw Middle east rate chauh emaw ni loin khawvel pum a nghawng dawn a ni," a ti.Iran chu dem chhunzawm leh bawkin an chetdan chu a atthlak hle tih a sawi. Jamal Khashoggi thihnaah mawh a phur ṭhen tih sawiMohammed bin Salman hi Pathianni khan CBS News chuan a interview a, hetah hian Iran chung chang a sawichhuah bakah Saudi journalist Jamal Khashoggi thah a nih chungchang a sawi bawk. He interview-naah hian Salman hi Jamal Khashoggi that turin thu a pe em tih zawhna an zawt a ni.Saudi crown prince chuan that tura thupe nia an puhna chu pha in, a hunlai khan engmah a hriat loh thu a sawi. Mahse Khashoggi thihnaah chuan Saudi Arabia ram hruaitu a nihnaṭangin mawh a phur tih sawiin, a bik takin a thattu chu Saudi sorkar hnuaia hnathawk a nih avangin mawhphurtu nia a inhriat thu a sawi a ni.Khashoggi hi nikum October 2 khan Turkey-a Saudi Arab...	passage: Saudi crown prince-in Iran khap Ṭul a ti Saudi crown prince Mohammed bin Salman chuan Iran chu khaptu an awm a nih loh chuan khawvel-a oil man chu nasa takin a sang thei dawn tih a sawi.Mohammed bin Salman chuan hmalakna a awm lo a nih chuan Iran chu a huangtau telh telh ang a, indona a chhuak thei mai dawn a ni, a ti. Chutiang a nih chuan khawvel economy-in nasa takin a tuar dawn niin a sawi bawk.Salman hian an oil facility pahnih beihnaah chuan Iran chu mawhphurtuah a puh ve bawk a, mahse Iran chuan Saudi crown prince thusawi chuan zahna bak engmah a thlen lo ang, tiin a chhanglet ve thung.Saudi lal la ni mai tur chuan khawvelin Iran lakah hma a la lo a nih chuan khawvel chuan harsatna nasa zawk a tawk dawn a ni tia sawiin, oil supply chu nasa takin a buai dawn bakah nasa takin a man chu a sang chho dawrh dawn a ni, a ti bawk. Middle East region chuan khawvel energy supply 30% zet an tum phak tih leh, global trade passage 20% leh khawvel pum GDP 4% chu an tum phak tih a sawi...
  query: Mizoram sorkarin journalist-te a ngai pawimawh tih tuin nge sawi?	passage: Sorkarin journalist-te a ngai pawimawh - Ruatkima MJA chuan nimin khan Aizawl Press Club-ah tun hnaia Parliament House-a training-a kal an member mi 15 leh IPR department officer pangate pualin zin report pek inkhawm an buatsaih a; khuallian, IPR minister Lalruatkima chuan, Mizoram sorkarin thuthar thehdarhtu (journalist)-te a ngaih pawimawh thu a sawi.IPR Minister chuan, chief minister Zoramthanga'n journalist-te a ngaih pawimawh thu sawiin, "Tun ṭum zinna atan leh journalist-te pawimawh dang atan Chief Minister hnenah Finance changtu a nih angin pawisa kan dil a, ani chuan 'Journalist-te chu an pawimawh a, anni tana kan tih ve theihah chuan theihtawp kan chhuah tur alawm' a ti a ni. Sorkarin thuthar thehdarhtute pawimawhzia a hria a, a ngai pawimawh a ni," a ti.Sorkar thu leh hla mipuite hriattir chu journalist-te hna pakhat a nih thu sawiin Lalruatkima chuan, "Sorkar kal dik lohnaah pawh min sawisel a, kawng dik min kawhhmuhtu tur in ni," a ti a; journalist-te chu hleih nei...	passage: Mizoram leh phai lam journalist-te inkâwm: Mizoram sorkar aiawhte'n biak an harsat ṭhin thu sawi chhuak Mizoram leh Assam inkara buaina thleng a hmuna zir chiang tura rawn kal phai lama journalist-te chu nimin khan Aizawl Press Club-ah MJA huaihawtin Mizo journalist-te nen an inkâwm. Mizoram chanchin thar inhrilh tawnnaa thawhhona leh inpawhna ṭha zawk a awm theih dan turte leh pawimawh dangte an sawiho a; phai lam journalist-te chuan Mizoram sorkar aiawhte biak pawh an harsat ṭhin thu an sawi chhuak. Inkawmhonaah hi Mizoram Journalists' Association Gen.Hqrs president Zonunsanga Khiangte-in a kaihruai a; tun hnaia Mizoram-Assam ramria thil thlenga thil thlen dan dik tak hre chiang tura a buzáwl ngeia luhchilhtu national media-te chu Mizo journalist-te'n an lo lawm tih sawiin, "Mizoram hi ram kilkhâwra awm kan nih avangin a ram leh a chhunga chêngte chanchin hi ram chhung hmun danga media lian zawkte'n an hre chiang tâwk lo fo a, hei hi tuna national media rawn kalte'n an rawn ...
  query: Thawhlehni khan Mali sorkar leh helho an inbeihnaah hian sipai engzat nge thi?	passage: Mali inbeihna thlengah mi 25 thi Mali sorkar chuan Thawhṭannia firfiakte beih an tawhnaah khan an sipai 25 an thih bakah midang 60 chu chin hriat lohin an awm tih an sawi. Thawhṭanni khan Mali-a Boulkessy leh Mondoro-a army outpost-te chu firfiakte hian an bei a ni.Sorkar chuan an sipaite chuan firfiak 15 an kap hlum ve tih sawiin, an ralthuam erawh an hloh nual tih an sawi bawk. Beih an tawhna hmunah hian Burkina Faso leh French force ṭangkawpte chuan beihpui an thlak chhunzawm a ni.Mali hian 2012-a Islamist firfiakten an ram hmar lam an thununa France-in a sipaite a tirhluh aṭang khan jihadist-te aṭangin beih an tawk fo a, tualchhunga hnam hrangte inkarah buaina a chhuak fo bawk.	passage: France airstrike-ah firfiak 50 chuang thi France sorkar chuan an sipaiten central Mali-a airstrike an neihah Al-Qaida nena inzawm, firfiak 50 chuang an thi tih a sawi. Sorkar thuchhuah chuan sipaite hian kar liam ta Zirtawpni khan Burnina Faso leh Niger ramri bulah sipaite hian Islamic firfiakte beihpui an thlak tih a tarlang. France defence minister Florence Parly chuan Mali sorkai aiawhte nena meeting an neih hnuah sipaite beihpui thlak, Operation Barkhane tih hi neih a nih thu sawiin, firfiak 50 chuang an thih bakah an ralthuam eng emaw zat mansak an ni bawk a ni, a ti. Beihpui thlaknaah hian motorcycle 30 vel tihchhiah a ni tih a sawi bawk. Sipai thupuangtu Colonel Frederic Barbry chuan firfiak pali man an ni bawk tih sawiin, firfiak an beite hian sipai awmna hmun beih an tum a ni, a ti. Barbry chuan Greater Sahara-ah pawh beihpui thlak mek a ni tih sawiin, sipai 3,000 vel an tel tih a sawi.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• optim: adamw_torch_fused
• gradient_accumulation_steps: 8
• bf16: True
• data_seed: 42
• warmup_ratio: 0.0

All Hyperparameters

• per_device_train_batch_size: 8
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 8
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: 42
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: 0.0
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 3.9 hours

Framework Versions

• Python: 3.10.20
• Sentence Transformers: 5.5.1
• Transformers: 5.9.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.14.0
• Datasets: 4.8.5
• Tokenizers: 0.23.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}