SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("rookshanks/gemma_300m_grocery")
# Run inference
queries = [
    "miracle reds superfood",
]
documents = [
    'Categories: Pantry > Canned & Pickled  > Natural and Organic, Natural and Organic > Canned > Fruit and Vegetables\nBrand: Muir Glen\nProduct: Fire Roasted Crushed Tomatoes\nSize: 796 ml\nDescription: Whole tomatoes are harvested at peak flavour, roasted over an open flame that blackens the skin and sears in the unique flavour of fire-roasted goodness. Then they are crushed for easy use in soups, sauces and stews. 1/2 cupA gluten-free foodOrganicNon GMO',
    'Category: Fruits & Vegetables > Fresh Vegetables > Carrots, Radish & Root Vegetables\nBrand: none\nProduct: Red Carrot\nSold by weight\nDescription: A sweet, earthy flavour with a satisfyingly crisp texture in every bite. Shred these carrots raw into salads for a bright crunch, or roast them whole to deepen their natural sweetness for a simple side dish. They also add a foundational flavour and colour to hearty soups and stews.',
    "Category: Snacks, Chips & Candy > Chips & Snacks > Snack & Granola Bars\nBrand: Kashi\nProduct: Chewy Chia Whole Grain Bars Dark Chocolate, Almond & Sea Salt\nSize: 175 g\nDescription: Our chewy granola bars are truly lovable. They bring our unique blend of whole grains together with whole, roasted nuts and succulent dark chocolate. And with 4g of fibre per serving, they are as nourishing and tasty.Combining dark chocolate, roasted almonds and a touch of sea salt with chia means you can enjoy sweet and salty flavour with 0.1 g omega-3 polyunsaturates in one satisfyingly chewy granola bar.A sweet and salty granola bar made with whole grain oats, crunchy almonds, rich dark chocolate and a touch of sea salt.Chewy, Tasty, Simple. Kashi Chewy Whole Grain Bars arethe perfect snack for the whole family. Great for snacking at home, at school, at the office, or when you're on the go!13g Whole Grains per serving per serving; 4g fibre per servingNo artificial flavours or coloursNon-GMO Project Verified",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.2978, 0.3890, 0.0984]])

Evaluation

Metrics

Triplet

• Dataset: val_concordance
• Evaluated with TripletEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,571 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 3 tokensmean: 5.32 tokensmax: 12 tokens	min: 21 tokensmean: 133.25 tokensmax: 531 tokens	min: 21 tokensmean: 121.28 tokensmax: 509 tokens

• Samples:

  anchor	positive	negative
  waffles	Category: Frozen Food > Bakery & Breakfast > Waffles & PancakesBrand: President's ChoiceProduct: Gluten Free Homestyle WafflesSize: 320 gDescription: Crisp, golden, and naturally flavoured with a hint of vanilla, Original Waffles are a gluten-free breakfast favourite. Light and delicious, they’re perfect toasted and topped with fruit, syrup, or your go-to spread.	Category: Frozen Food > Bakery & Breakfast > Waffles & PancakesBrand: No NameProduct: Blueberry WafflesSize: 280 gDescription: Start your day with the sweet, fruity taste of naturally flavoured Blueberry Waffles. Light, crisp, and golden, they’re perfect toasted and topped with your favourite syrup or fresh fruit. It makes your mornings quick and delicious.
  hot sauce for wings	Category: Pantry > Condiments & Sauces > Hot & Chili SaucesBrand: TabascoProduct: Tabasco Hot SauceSize: 57 mlDescription: TABASCO Brand Pepper Sauce..Our classic creation - always the first choice of hot sauce lovers around the world.TABASCO.This is the Genuine Article.Since 1868.TABASCO Brand Pepper Sauce is made the same today as it was by Edmund McIlhenny. The red TABASCO peppers, mashed with a small amount of Avery Island salt and aged for up to 3 years..Bold Flavour range.Tabasco Green Sauce.Tabasco Garlic Sauce.Tabasco Chipotle Sauce.Original Tabasco Sauce.Tabasco Hanabero Sauce.Milder ? Hotter	Category: Meat > Chicken & Turkey > Chicken WingsBrand: noneProduct: Chicken Wing Splits Club Pack, HalalSold by weightDescription: Get ready for game day or a weekend gathering. These Halal chicken wing splits are conveniently separated, saving you prep time. Bake them in the oven until the skin is golden and crisp, or toss them in a spicy barbecue sauce for a sticky, satisfying main course. Serve with celery sticks and a creamy blue cheese dip.
  pop-tarts churro	Category: Pantry > Cereal & Breakfast > Toaster PastriesBrand: KelloggsProduct: Poptarts Pop Tarts Frosted SmoreSize: 384 gDescription: Toasted, frozen, or straight from the box ; add a little fun to your mornings with Pop-Tarts S'mores toaster pastries.A delicious graham cracker flavoured crust stuffed with chocolatey marshmallow filling, frosted with rich chocolatey icing. A chocolatey & fun way to awaken your senses at breakfast.A source of 4 essential nutrients185 calories per pastry (48 g)	Category: Snacks, Chips & Candy > Chips & Snacks > ChipsBrand: OrionProduct: Snack Chip Chocolate Flavor Choco ChurrosSize: 160 g

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,143 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 3 tokensmean: 5.44 tokensmax: 12 tokens	min: 21 tokensmean: 130.98 tokensmax: 618 tokens	min: 19 tokensmean: 121.34 tokensmax: 513 tokens

• Samples:

  anchor	positive	negative
  broccoli florets	Category: Fruits & Vegetables > Fresh Cut Fruits & Vegetables > Fresh Cut VegetablesBrand: President's ChoiceProduct: Broccoli FloretsSize: 907 gDescription: These fresh-tasting, premium cut broccoli florets are packed for your convenience.	Categories: Fruits & Vegetables > Fresh Vegetables > International & Specialty Vegetables, International Foods > East Asian Foods > Fruits & VegetablesBrand: noneProduct: BroccoliniSize: 1 eaDescription: Broccolini (or sweet baby broccoli) is a cross between broccoli and Chinese broccoli. It has slender bright green stalks topped with small florets and a sweet, peppery flavour. Add raw broccolini to your next vegetable platter or blanch and toss with butter and lemon juice for a simple side dish.
  organic strawberry spread for toast	Category: Natural and Organic > Cereals, Spreads & Syrups > SpreadBrand: Crofter'sProduct: Organic Seedless Raspberry SpreadSize: 383 mlDescription: Pick the Best Crofter's Premium Spread Organic - Est. 1989	Category: Natural and Organic > Cereals, Spreads & Syrups > SpreadBrand: PB&MeProduct: Powdered Peanut Butter Size: 453 gDescription: All natural. Gluten free. 5 g protein. Just add water!. 90% less calories from fat than traditional peanut butter
  balsamic vinaigrette for salad	Category: Pantry > Condiments & Sauces > Salad DressingBrand: ReneesProduct: Balsamic VinaigretteSize: 350 mlDescription: Renée’s vinaigrette delivers an exceptional homemade flavour that will complement your meals! Our balsamic vinaigrette is sweet and has no artificial colours or flavours. Made with authentic Modena balsamic vinegar and canola oil, Renée’s Balsamic Vinaigrette is perfect for your chicken dinner or a decadent salad. You can even use our vinaigrette as a delightful dipping sauce for vegetables. Renée’s salad vinaigrette comes in a convenient bottle and should be refrigerated. Made with quality ingredients, our vinaigrette is proud to expire. Our gourmet products are prepared in Montreal, Canada.One 350 mL bottle of Renée’s Balsamic VinaigretteRenée’s Balsamic Vinaigrette has a delicious tart and sweet tasteOur balsamic vinaigrette is made with authentic Modena balsamic vinegarMade with no artificial flavours or colours, our balsamic vinaigrette is great for a varie...	Category: Pantry > Canned & Pickled > Pickled & Marinated VegetablesBrand: PC Black LabelProduct: Castelvetrano OlivesSize: 375 ml

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• bf16: True
• prompts: {'anchor': 'query', 'positive': 'document', 'negative': 'document'}

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: {'anchor': 'query', 'positive': 'document', 'negative': 'document'}
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Framework Versions

• Python: 3.12.4
• Sentence Transformers: 5.3.0
• Transformers: 4.57.6
• PyTorch: 2.10.0
• Accelerate: 1.13.0
• Datasets: 4.8.4
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}