SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '41 year old man with history of severe intellectual disability, CHF, epilepsy presenting with facial twitching on the right and generalized shaking in at his NH which required 20 mg valium to cease seizure activity. Per outside medical patient was felt to have focal epilepsy with secondary generalization, likely due to anoxic brain injury at birth, and probably related to the atrophic changes seen on MRI, particularly in the left temporal lobe.\nThe patient first developed seizures at age 13 found by family to have a generalized convulsion. He had a second seizure two years after his first episode. He was maintained on Dilantin and phenobarbital. The patient went 20 years without another seizure. He was recently tapered off Dilantin, and it was felt that perhaps this medication was necessary to maintain him seizure free. The patient had no further events during the hospital course and was back at his baseline at the time of discharge. Full EEG reports are pending at the time of dictation.\nPast Medical History:\nEpilepsy as above, CHF, depression',
    'Mesial temporal lobe epilepsy (MTLE) is the most common cause of medication-resistant\r\n      epilepsy in adults. The standard treatment for refractory MTLE is surgical resection by\r\n      craniotomy. Stereotactic laser interstitial thermal therapy (LITT) is a new surgical\r\n      technique being used to treat MTLE. Under MRI-guidance, a laser probe is inserted into the\r\n      seizure focus and heat is used to destroy the tissue. Compared to temporal lobectomy, LITT\r\n      results in shorter hospital stays, low complication rates, and possibly less cognitive\r\n      decline; however, seizure freedom rates are potentially lower.\r\n\r\n      During temporal lobectomy, neurophysiologic intraoperative monitoring (NIOM) can be used to\r\n      better identify epileptogenic tissue and guide resection. This tool has been unavailable\r\n      during LITT procedures. Recently, the investigators demonstrated in two cases that NIOM with\r\n      a depth electrode is technically feasible during LITT and can identify epileptiform activity\r\n      intra-operatively.\r\n\r\n      This is a prospective trial of NIOM during LITT for mesial temporal lobe epilepsy. The\r\n      investigators will assess the safety of performing NIOM during LITT and whether data from\r\n      NIOM (frequency and characteristics of epileptiform discharges recorded before and after\r\n      ablation) are associated with seizure outcomes. If there is an association, NIOM could be\r\n      used for prognostication and could potentially even be used to guide surgery.\r\n\r\n      Hypotheses:\r\n\r\n        1. NIOM performed by MRI-guided stereotactic depth electrode placed in the parahippocampal\r\n           gyrus adjacent to the LITT catheter is safe, as compared to institutional LITT controls\r\n           without NIOM and published LITT complication rates.\r\n\r\n        2. Greater magnitude fractional decrements in discharge frequency from pre-ablation to\r\n           post-ablation recordings will be significantly associated with better seizure outcomes,\r\n           as measured by International League Against Epilepsy (ILAE) surgical outcome scores.\r\n\r\n      Objectives:\r\n\r\n      The primary project goals are to assess if NIOM by parahippocampal depth electrode is safe\r\n      during LITT of MTLE and to assess if the fractional decrement of interictal discharges (ID)\r\n      on NIOM can be significantly correlated with outcome. The study will be powered to address\r\n      these questions a priori. Post hoc analyses consisting of a multivariate analysis of other\r\n      patient demographic data, NIOM findings, operative parameters, quality of life scores, and\r\n      neuropsychiatric outcomes will also be assessed.',
    "BACKGROUND There is no consensus regarding the injury mechanism in complex prolonged Whiplash\r\n      Associated Disorders (WAD) cases. Often, tissue damage and physiological alterations is not\r\n      detectable. In order to improve future rehabilitation, a greater understanding of the\r\n      mechanisms underlying whiplash injury and their importance for treatment success is required.\r\n      It is also important to investigate if pathophysiological changes can be restored by\r\n      rehabilitation.\r\n\r\n      AIM The projects aims to investigate neck muscle structure and function, biomarkers and the\r\n      association with pain, disability and other outcomes before and after neck-specific\r\n      exercises.\r\n\r\n      METHODS Design These are sub-group trials, each one independent of the others, in a\r\n      prospective, multicentre, randomized controlled trial (RCT) with two parallel treatment arms\r\n      conducted according to a protocol established before recruitment started (ClinicalTrials.gov\r\n      Protocol ID: NCT03022812). Physiotherapist-led neck-specific exercise previously shown to be\r\n      effective for the current population constitutes the control treatment for the new\r\n      Internet-based neck-specific exercise treatment. In the RCT, a total of 140 patients will be\r\n      included (70 from each group), whereof 30 (both randomization arms equally) consecutively\r\n      will be asked to participate in the present sub-group study. The sub-group studies aims to\r\n      start September 2019. Independent physiotherapists in primary health care will distribute the\r\n      treatment.\r\n\r\n      In sub-group of individuals, additional measurements will be performed before and after\r\n      interventions end (3 months follow-up). The physical measurements will be performed by\r\n      independent specially trained test-leaders blinded to randomization.\r\n\r\n      Additionally, 30 neck healthy individuals without serious diseases matched for age and gender\r\n      will consecutively be investigated.\r\n\r\n      Study population\r\n\r\n      The inclusion criteria for patients are:\r\n\r\n        -  Chronic neck problems corresponding to WAD grades 2-3 verified by clinical examination\r\n\r\n        -  Average estimated pain in the last week at least 20 mm on the visual analogue scale\r\n           (VAS)\r\n\r\n        -  Neck disability of more than 20% on the Neck Disability Index (NDI) [10]\r\n\r\n        -  Working age (18 - 63 years)\r\n\r\n        -  Daily access to a computer/tablet/smart phone and Internet\r\n\r\n        -  Neck symptoms within the first week after the injury (i.e., neck pain, neck stiffness,\r\n           or cervical radiculopathy).\r\n\r\n      For the present sub group study additional criteria were:\r\n\r\n        -  Right handed\r\n\r\n        -  Dominant right sided or equal sided pain\r\n\r\n      Inclusion criteria for healthy controls:\r\n\r\n      • Age and gender matched healthy individuals without neck pain and disability (VAS <10mm, NDI\r\n      <5%) that feel overall healthy without known diseases.\r\n\r\n      Exclusion criteria for patients:\r\n\r\n        -  Individuals with any of the following signs of head injury at the time of whiplash\r\n           injury will be excluded: loss of consciousness, amnesia before or after the injury,\r\n           altered mental status (e.g., confusion, disorientation), focal neurological changes\r\n           (changes in smell and taste).\r\n\r\n        -  Previous fractures or dislocation of the cervical spine\r\n\r\n        -  Known or suspected serious physical pathology included myelopathy,\r\n\r\n        -  Spinal tumours\r\n\r\n        -  Spinal infection\r\n\r\n        -  Ongoing malignancy\r\n\r\n        -  Previous severe neck problems that resulted in sick leave for more than a month in the\r\n           year before the current whiplash injury\r\n\r\n        -  surgery in the cervical spine\r\n\r\n        -  Generalized or more dominant pain elsewhere in the body\r\n\r\n        -  Other illness/injury that may prevent full participation\r\n\r\n        -  Inability to understand and write in Swedish\r\n\r\n      Additional criteria in the present sub group:\r\n\r\n        -  Increased risk of bleeding,\r\n\r\n        -  BMI >35\r\n\r\n        -  Contraindications of MRI such as metal, severe obesity, pacemaker and pregnancy.\r\n\r\n      Exclusion criteria for healthy controls:\r\n\r\n        -  Earlier neck injury,\r\n\r\n        -  Recurrent neck pain,\r\n\r\n        -  Earlier treatment for neck pain.\r\n\r\n        -  Increased risk of bleeding,\r\n\r\n        -  BMI >35\r\n\r\n        -  Contraindications of MRI\r\n\r\n      Recruitment and randomization Information about the study will be provided by healthcare\r\n      providers, reports in newspapers, social media, and the university's website. Interested\r\n      patients will contact the research team through the project website. After completing a small\r\n      survey on the website, a project team member (physiotherapist) will perform a telephone\r\n      interview and ask about the patient's medical history. An appointment for a physical\r\n      examination and additional interview for the present sub-group study is made as a last step\r\n      to ensure that the criteria for study participation are met. If the study criteria are met,\r\n      written and oral informed consent are obtained, and the patient will fill out a questionnaire\r\n      and undergo physical measurements of neck-related function. Baseline measurements must be\r\n      completed for inclusion.\r\n\r\n      Healthy individuals will consecutively be recruited among friends, family and staff at the\r\n      university or the university hospital to suit the age and gender of a patient.\r\n\r\n      Intervention for the patient group The intervention consists of neck-specific exercises\r\n      distributed in two different ways, twice a week at the physiotherapist clinic for 3 months\r\n      (NSE group) or with 4 physiotherapy visits only combined with a web-based system (NSEIT\r\n      group).\r\n\r\n      A. In the NSE group, patients will get an explanation and justification for the exercise\r\n      consisting of basic information about the musculoskeletal anatomy of the neck relevant to the\r\n      exercises given by the physiotherapist in order to motivate the patient and help make them\r\n      feel safe and reassured. The patients undergo a 12-week training programme with a\r\n      physiotherapist 2 days/week (total 24 times). Exercises are chosen from a clear and written\r\n      frame of exercises. The training includes exercises for the deep neck muscles, continuing\r\n      with the endurance training of neck and shoulder muscles. The exercises are individually\r\n      adjusted according to the individual's physical conditions and progressively increased in\r\n      severity and dose. Exercise-related pain provocation is not accepted. The patient may also\r\n      perform exercises at home. At the end of the treatment period, the participants are\r\n      encouraged to continue practising on their own. The exercises have been used with good\r\n      results in previous RCTs.\r\n\r\n      B. In the NSEIT group, patients will receive the same information and training programmes as\r\n      the NSE group, but with 4 visits to the physiotherapist instead of 24. Exercises are\r\n      introduced, progressed, and followed up to ensure correct performance. The exercises are\r\n      performed and most of the information is given with the help of Internet support outside the\r\n      healthcare system. Photos and videos of the exercises (a clear stepwise progression) and\r\n      information are available on the Web-based system. A SMS reminder is automatically available\r\n      if the exercise diary is not completed. The time required for training is the same as in\r\n      group A, but without the patient having to go to the physiotherapy clinic. The Internet\r\n      programme was developed by experienced physiotherapists/ researchers together with\r\n      technicians and clinicians. Technicians are available to assist the participants if any\r\n      technical difficulties arise. The patients will be introduced to the exercises and get\r\n      information and support at the physiotherapy visits.\r\n\r\n      Variables and measurements Background data and data in the RCT include personal details,\r\n      questionnaires and test results regarding pain, physical and psychological functioning,\r\n      health and cost-effectiveness described elsewhere (ClinicalTrials.gov Protocol ID:\r\n      NCT03022812).\r\n\r\n      Measurements will be done at baseline for both groups and at 3 months follow-up for the\r\n      patient when treatment ends. Except for blood and saliva samples that will be collected twice\r\n      (baseline and repeated after 3 months), the measurements will be performed at baseline only\r\n      for the healthy individuals.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 11,583 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 82 tokensmean: 214.67 tokensmax: 355 tokens	min: 4 tokensmean: 227.41 tokensmax: 384 tokens	min: 4 tokensmean: 259.12 tokensmax: 384 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  The patient is a 35-year-old woman with myasthenia gravis, class IIa. She complains of diplopia and fatigue and weakness that affects mainly her upper limbs. She had a positive anti-AChR antibody test, and her single fiber electromyography (SFEMG) was positive. She takes pyridostigmine 60 mg three times a day. But she still has some symptoms that interfere with her job. She is a research coordinator and has 3 children. Her 70-year-old father has hypertension. She does not smoke or use illicit drugs. She drinks alcohol occasionally at social events. Her physical exam and lab studies were not remarkable for any other abnormalities.BP: 110/75Hgb: 11 g/dlWBC: 8000 /mm3Plt: 300000 /mlCreatinine: 0.5 mg/dlBUN: 10 mg/dlBeta hcg: negative for pregnancy	Randomized, double-blind, placebo-controlled, parallel group study is designed to evaluate
  the safety, tolerability and efficacy of amifampridine phosphate in patients with MuSK-MG. In
  addition, a sample of AChR-MG patients will be assess for efficacy and safety of
  amifampridine phosphate. Planned duration of participation for each patient is at least 38
  days, excluding the screening period. Eligible patients will be titrated to an efficacious
  dose of amifampridine phosphate and those who demonstrate improvement will be randomized to
  either placebo or amifampridine, in a double-blind fashion, for 10 days.	this randomized controlled trial will compare the impact of routine use of completion
  angiography versus using it on selective bases after thromboembolectomy in patients with
  acute lower limb ischemia and their impact on limb salvage rates
  The patient is a 17-year-old boy complaining of severe migratory pain in the right lower quadrant of his abdomen that started four days ago. The pain is accompanied by nausea and vomiting. He was febrile with tenderness, rebound tenderness and guarding on palpation. His WBC was elevated with dominant neutrophils. CT scan showed evidence of acute perforated appendicitis with free fluid in the pelvis. Diagnostic laparoscopy revealed phlegmon with no other abdominal abnormalities. He is now a candidate for emergent laparoscopic appendectomy under general anesthesia.	Acute appendicitis is one of the most common causes of abdominal pain in emergency
  departments as well as one of the most common indications for emergency abdominal surgery.
  The clinical diagnosis has been based on patient history, physical examination and laboratory
  findings as well as the "clinical eye" of the surgeon. Still the diagnosis remains
  challenging. One of the main problems is that many other disorders can mimic the clinical
  presentation of appendicitis, thus increasing the role of imaging techniques to aid in
  diagnostic accuracy. Now preoperative imaging in patients with suspected acute appendicitis
  is currently widely accepted as the standard of practice, and CT has been shown to clearly
  outperform US in terms of diagnostic performance. Nowadays, CT imaging is considered the
  primary imaging modality in the diagnosis for acute appendicitis as it is appraised for its
  high sensitivity and specificity. The ...	Urticaria is a common skin disorder that is classified according to its chronicity into acute
  and chronic forms. It may occur spontaneously or on exposure to a physical factor. In the
  latter case, the urticaria is classified as a physical urticaria . Physical urticaria may be
  induced by mechanical and applied pressure, exercise, or exposure to cold, heat, sun, water,
  or vibration. The urticarial lesions are generally thought to be the result of mast cell
  activation and degranulation, which is supported by the finding of increased levels of serum
  histamine during some urticarial flares. Passive transfer experiments, whereupon serum from
  affected donors is transferred into recipient s skin followed by physical stimulation with
  resultant urticaria at the site of challenge, have been positive in some instances. This
  suggests the presence of an intrinsic factor in serum, such as IgE, which then mediates
  activation of tiss...
  34 year old woman with Marfan's syndrome and known severe mitral valve prolapse with regurgitation, who was planned for a MV repair but was lost to follow-up. She remains symptomatic and is now prepared to undergo mitral valve repair/replacement surgery. EF of 65% on TTE. Past Medical History:Marfans SyndromeMVP with severe mitral regurgitationGastric reflux diseaseHistory of gestational diabetes mellitusHypertension with pregnancyObesityc-section x 2laser eye surgerycataract surgeryfoot surgery (shorten bone length)	Early feasibility study - multi-center, prospective, single-arm, and non-randomized study
  without concurrent or historical controls.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 2
• per_device_eval_batch_size: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 2
• per_device_eval_batch_size: 2
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 3.4.1
• Transformers: 4.50.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}