SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("GPTasty/TastyRecipesEmbedder")
# Run inference
sentences = [
    'NAME: Spinach with Raisins and Pine Nuts\n\nCATEGORY: Fruit\n\nKEYWORDS: Vegetable, Nuts, Low Cholesterol, Healthy, < 30 Mins, Stove Top\n\nTOOLS: grill, pot\n\nINGREDIENTS: fresh spinach, pine nuts, salt, raisins, olive oil, lemon juice\n\nINSTRUCTIONS: \nClean the spinach thoroughly.\nGrill the pine nuts until golden brown, watching carefully so as not to burn.\nBring a pot of salted water to the boil and toss in raisins and spinach.\nDrain as soon as spinach goes limp.\ntoss in olive oil and lemon juice, and scatter with the grilled pine nuts.',
    'NAME: Dried Apricots with Pistachios and Almonds\n\nCATEGORY: Fruit\n\nKEYWORDS: Dried Fruit, Nuts, Low Cholesterol, Healthy, < 30 Mins, Stove Top, Vegan\n\nTOOLS: grill, pot\n\nINGREDIENTS: dried apricots, pistachios, salt, slivered almonds, olive oil, orange juice\n\nINSTRUCTIONS:\nSoak the dried apricots in warm water for 10 minutes to soften them.\nGrill the pistachios until lightly toasted, being careful not to burn them.\nBring a pot of salted water to the boil and add the softened apricots.\nDrain immediately after the apricots plump up slightly.\nToss with olive oil and orange juice, then sprinkle with the grilled pistachios and slivered almonds.',
    'NAME: Smoky Chipotle Turkey Meatloaf\n\nCATEGORY: Meat\n\nKEYWORDS: < 60 Mins, Spicy, Oven, Comfort Food\n\nTOOLS: frying pan, meat thermometer, oven, loaf pan\n\nINGREDIENTS: bacon, yellow onion, green bell pepper, chipotle powder, garlic powder, dried oregano, salt, ground mustard, smoked paprika, chili powder, tomato paste, chicken broth, eggs, ground turkey\n\nINSTRUCTIONS:\nPreheat oven to 425 degrees.\nCook bacon in frying pan, remove, drain, and chop.\nLeave drippings in pan and saute (but do not brown) onion and green pepper.\nAdd chipotle powder, garlic powder, oregano, salt, mustard, smoked paprika, and chili powder.\nCook for 8 minutes.\nRemove pan from heat and add tomato paste and chicken broth.\nMix bread crumbs with eggs and add to ground turkey.\nAdd spice mixture and bacon to turkey mixture and mix gently.\nPlace mixture in two or three 8 x 4 inch individual loaf pans.\nCook until done, about 35 to 45 minutes, or until internal temperature reaches 165 degrees on a meat thermometer.\nLet rest for 10 minutes before slicing.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 121,408 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 45 tokensmean: 185.8 tokensmax: 384 tokens	min: 59 tokensmean: 222.58 tokensmax: 384 tokens

• Samples:

  sentence_0	sentence_1
  NAME: Homemade Honey MustardCATEGORY: SaucesKEYWORDS: Low Protein, < 15 Mins, EasyTOOLS: INGREDIENTS: Dijon mustard, sour cream, honey, Worcestershire sauceINSTRUCTIONS: Mix well, enjoy.	NAME: Creamy Maple Mustard SauceCATEGORY: SaucesKEYWORDS: Low Protein, < 15 Mins, Easy, Gluten-FreeTOOLS:INGREDIENTS: Whole grain mustard, Greek yogurt, maple syrup, apple cider vinegarINSTRUCTIONS: Combine all ingredients in a bowl and mix until well combined. Refrigerate for at least 10 minutes before serving to allow flavors to meld. Enjoy with pretzels or veggies.
  NAME: Baby Greens With Hazelnut Parmesan CrispsCATEGORY: GreensKEYWORDS: Vegetable, High In..., < 30 MinsTOOLS: parchment paper, mixer, whisk, oven, baking sheetINGREDIENTS: parmesan cheese, hazelnuts, lemon juice, olive oil, maple syrup, lettuce, prosciuttoINSTRUCTIONS: Preheat oven to 350°F Line a baking sheet with parchment paper.Combine Parmesan and hazelnuts. Drop 12 spoonfuls of Parmesan mixture onto baking sheet 3 inches apart.Bake crisps for 8 to 10 minutes, or until golden. Cool on baking sheet.Whisk together lemon juice, oil and maple syrup. Season with salt and pepper.Toss lettuce with vinaigrette and pile on individual plates.Coil each slice of prosciutto into a rose shape and set a rose in center of each mound of greens. Garnish each serving with two Parmesan crisps.	NAME: Spinach Salad with Almond Manchego CrispsCATEGORY: GreensKEYWORDS: Vegetable, High In..., < 30 Mins, Gluten-FreeTOOLS: parchment paper, mixer, whisk, oven, baking sheetINGREDIENTS: manchego cheese, almonds, lime juice, avocado oil, honey, spinach, serrano hamINSTRUCTIONS:Preheat oven to 375°F. Line a baking sheet with parchment paper.Combine Manchego cheese and chopped almonds. Drop 12 spoonfuls of the Manchego mixture onto the baking sheet, spacing them 3 inches apart.Bake crisps for 6 to 8 minutes, or until golden brown. Let cool on the baking sheet.Whisk together lime juice, avocado oil, and honey. Season with salt and a pinch of red pepper flakes.Toss spinach with the vinaigrette and arrange on individual plates.Roll each slice of serrano ham into a flower shape and place one in the center of each spinach mound. Garnish each serving with two Manchego crisps.
  NAME: Classic Delicious New York CheesecakeCATEGORY: CheesecakeKEYWORDS: Dessert, Weeknight, For Large Groups, < 4 HoursTOOLS: pan, mixing bowl, warm oven, mixer, refrigeratorINGREDIENTS: graham cracker crumbs, cream cheese, eggs, sour cream, butter, sugar, vanillaINSTRUCTIONS: Preheat oven to 450 degrees.To make the crust, mix graham crackers crumbs, butter, and 2 tablespoons of sugar in bowl.Press mixture in bottom and sides of 9 inch springform pan.In mixing bowl, beat cream cheese and remaining sugar for 2 minutes.Add eggs and vanilla to mixture and mix until well blended.Then stir or fold in sour cream.Pour mixture in crust filled pan and bake for 10 minutes.Then reduce to 200 degrees to bake for 45 minutes.From here the cheese cake just needs to be chilled, but I recommend doing the following step if you have a few extra hours- Leave in warm oven, once you turn it off but leave door slightly open.Let sit and cool for 2 hours and remove from oven.Remove sides ...	NAME: Lemon Ricotta Cheesecake DelightCATEGORY: CheesecakeKEYWORDS: Dessert, Weeknight, For Large Groups, < 4 Hours, CitrusTOOLS: pan, mixing bowl, warm oven, mixer, refrigerator, zesterINGREDIENTS: gluten-free graham cracker crumbs, ricotta cheese, eggs, Greek yogurt, butter, sugar, vanilla extract, lemon zest, lemon juiceINSTRUCTIONS:Preheat oven to 450 degrees Fahrenheit.To make the crust, mix gluten-free graham cracker crumbs, melted butter, and 2 tablespoons of sugar in bowl.Press mixture firmly in bottom and partially up the sides of a 9 inch springform pan.In a large mixing bowl, beat ricotta cheese and remaining sugar for 3 minutes until light and fluffy.Add eggs, vanilla extract, lemon zest, and lemon juice to mixture; mix until just combined. Avoid overmixing.Gently fold in Greek yogurt.Pour mixture into the prepared crust-lined pan and bake for 12 minutes.Reduce oven temperature to 225 degrees Fahrenheit and continue baking for 40 minutes, or until the edge...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• fp16: True
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 4.0.1
• Transformers: 4.50.2
• PyTorch: 2.4.0
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}