SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the dog-breed-ntuple-retrieval dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • dog-breed-ntuple-retrieval

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dorrito-dev/all-MiniLM-L6-v2-dog-breed-retriever")
# Run inference
sentences = [
    'What is the temperament of the Belgian Malinois?',
    'Belgian Shepherds are known to be highly intelligent, alert and sensitive; they are typically highly trainable, vigilant and hard-working with a strong guarding instinct making them protective of property and family, and very well suited for service with security services. The Groenendael and Tervueren varieties have a reputation for occasionally being snappy, making them less suitable as companion dogs for children; the Laekenois, whilst considered very good with children, can occasionally be troublesome with other dogs. The Belgian Shepherd responds well to training, and requires training ideally from an early age. The breed is very active – particularly the Malinois, which may reflect its continued breeding for security roles. While Belgian Shepherds of all coat types require ample exercise, the breed adapts well to living indoors.',
    'Indian Spitzes are small dogs - around 33 cm (13 in) at the withers - with a soft chest, fair tail, and a milky white double coat. Their eyes have visible irises that can be green or blue, giving them an unusually expressive face. The original Spitz was bred for hunting in cold conditions, and their white coloring reflects this. Although the same conditions are not found in India, their coloring is attractive and has remained through the years. Some can be white and brown.  Pointy fox-like ears make this breed especially expressive. Thick fur covers the outside of the ears and much of the inside as well, requiring extra attention when it comes to care and grooming. The tail of an Indian Spitz curls over their back and is quite fluffy. Their legs are only slightly longer than their bodies, which makes their heads look large. Indian Spitzes may be one of the easiest dogs to live with, making them one of the most popular family dog among Indian breeds. They are easily housebroken and trained. Grooming and exercise are relatively low maintenance, and they are highly adaptable dogs. They are equally comfortable in small apartments and in large, open houses. However, they do need moderate exercise.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6201, 0.2236],
#         [0.6201, 1.0000, 0.1353],
#         [0.2236, 0.1353, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: dog-breed-ir
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

dog-breed-ntuple-retrieval

• Dataset: dog-breed-ntuple-retrieval
• Size: 290 training samples
• Columns: anchor, positive, negative_1, negative_2, negative_3, negative_4, and negative_5
• Approximate statistics based on the first 290 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5
  type	string	string	string	string	string	string	string
  details	min: 8 tokensmean: 12.2 tokensmax: 16 tokens	min: 30 tokensmean: 195.52 tokensmax: 256 tokens	min: 42 tokensmean: 179.0 tokensmax: 256 tokens	min: 37 tokensmean: 190.59 tokensmax: 256 tokens	min: 37 tokensmean: 188.29 tokensmax: 256 tokens	min: 37 tokensmean: 190.62 tokensmax: 256 tokens	min: 42 tokensmean: 188.54 tokensmax: 256 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5
  Tell me about the summary of the Labrador Retriever.	The Labrador Retriever, also known simply as the Labrador or Lab, is a British breed of retriever gun dog. It was developed in the United Kingdom from St. John's water dogs imported from the Colony of Newfoundland, and was named after the Labrador region in that colony. It is among the most commonly kept dogs in several countries, particularly in the Western world. Labradors are often friendly, energetic and playful. It was bred as a sporting and hunting dog, but is widely kept as a companion dog. Though content as a companion, these dogs are intelligent and require both physical and mental stimulation. It may also be trained as a guide or assistance dog, or for rescue or therapy work. In the 1830s the 10th Earl of Home and his nephews, the 5th Duke of Buccleuch and Lord John Scott, imported progenitors of the breed from Newfoundland to Europe for use as gun dogs. Another early advocate of these Newfoundland fishing dogs was the 2nd Earl of Malmesbury, who bred them for their expertise...	They are natural hunters with an excellent ability to take training. Not only are they great pointers, but they are excellent retrievers as well. They will retrieve on land and in the water making the most of their natural instincts. However, they must be trained gently and without harsh commands or strong physical correction, as they have sensitive temperaments and can be easily damaged if trained too harshly. Vizslas thrive on attention, exercise, and interaction. They are intelligent dogs and need a lot of mental stimulation when young. If left alone for long hours, they can be bored and become destructive. With proper socialization and training, Vizslas are very gentle dogs that can be great around children. The Vizsla wants to be close to its owner as much as possible, and it is commonly observed that Vizslas insist on sleeping under the covers in their owner's bed at night.	The Bichon Frisé is a small dog, standing 23–30 cm (9–12 in) at the withers and weighing approximately 5 kg (11 lb), the weight varying in proportion to the height. The skull is rather flat, but may appear rounded; the muzzle tapers only slightly, and constitutes two-fifths of the length of the head. The nose is black, the eyes dark and round. The coat is loosely curled in spirals or corkscrews, with a thick soft undercoat. On the Bichon Frisé's head, it is typically styled in a spherical shape, resembling a full moon. The coat is always pure white; only in dogs under a year old may it be slightly tinged with beige, this extending over no more than 10% of the area of the body. The head and legs are proportionate in size to the body.	The breed is highly affectionate, playful, extremely patient and eager to please. As such, dogs of the breed are good with children and other dogs. Cavaliers are not shy about socialising with much larger dogs. They will adapt quickly to almost any environment, family, and location and suit city and country life. Their ability to bond with larger and smaller dogs makes them ideal in houses with more than one breed of dog as long as the other dog is trained. Cavaliers rank 44th in Stanley Coren's The Intelligence of Dogs, being of average intelligence in working or obedience. Cavaliers are naturally curious and playful, but also enjoy simply cuddling up on a cushion or lap, making them excellent companion or lap dogs for medical patients and the elderly. Cavaliers are active and sporting. Cavaliers are successful in conformation shows, obedience and agility and they also make wonderful therapy dogs due to their sweet, gentle natures. The breed is adaptable in their need for exercise, ha...	The general appearance of the Shetland is that of a miniature Rough Collie. They are a small and double coated dog with a well shaped head. They have a long coat which becomes short around the face giving it the appearance of a mane.	A 2024 UK study found a median life expectancy (95% CI) of 13.2 (13.1–13.2) years for the breed compared to an average of 12.7 for purebreeds and 12 for crossbreeds. A 2024 Italian study found a life expectancy of 10 years for the breed compared to 10 years overall. A 2005 Swedish study found that 22% of Golden Retrievers died by the age of 10, less than the overall rate of 35% of dogs dying by the age of 10. The breed is unusually prone to cancer, with one United States study finding cancer to be the cause of death in approximately 50% of the population, the second highest in the study. Several European studies found a much lower prevalence (20–39%), which may reflect the significant genetic divergence between the American and European populations. They are especially prone to haemangiosarcoma and lymphoma, with an estimated lifetime risk of one in five for the former and one in eight for the latter. The breed is also the subject of the Golden Retriever Lifetime Study, a long-term pr...
  What is the history of the Labrador Retriever breed?	The Labrador breed dates back to at least the 1830s, when St John's water dogs bred by European settlers in Newfoundland were first introduced to Britain from ships trading between Canada and Poole in Dorset. These were then bred with British hunting dogs to create what became known as the Labrador Retriever. Its early patrons included the Earl of Malmesbury, the Duke of Buccleuch, the Earl of Home and Sir John Scott. Early writers have confused the Labrador with the much larger Newfoundland and the Lesser Newfoundland, with Charles William George St John even referring to the Lesser Newfoundland as the Newfoundland. Colonel Peter Hawker describes the first Labrador as being not larger than an English Pointer, more often black than other colours, long in its head and nose with a deep chest, fine legs, and short and smooth coat, and did not carry its tail as highly as the Newfoundland. Hawker distinguishes the Newfoundland from both the "proper Labrador" and St John's breed of these dog...	The Brittany Spaniel or Breton Spaniel (French: Épagneul Breton) is a French breed of gun dog of spaniel type, bred primarily for bird hunting. It developed between the seventeenth and nineteenth centuries in Brittany in north-west France, and was officially recognized early in the twentieth. The breed has developed differently in the United States, where it may be referred to as the 'American Brittany'.	The German Shorthaired Pointer or Deutsch Kurzhaar is a German breed of continental pointing dog of Braque type. It originated in the nineteenth century in what is now Germany. It is of medium size, and is an all-purpose gun dog suitable for hunting and retrieving on both land and water. It may also be kept as a companion dog.	The Rhodesian Ridgeback is a large dog breed originally bred in Southern Africa. The original breed standard was drafted by F.R. Barnes, in Bulawayo, Southern Rhodesia (now Zimbabwe), in 1922, and approved by the South African Kennel Union in 1927. Its forebears can be traced to the ridged hunting and guardian dogs of the Khoikhoi. These were interbred with European dogs by the early colonists of the Cape Colony for the baying of lions.	The Khoikhoi people who lived in the Cape Peninsula when the Dutch began trading with the area during the mid 17th century, had a hunting dog which was described by Europeans as absolutely fearless and ferocious when acting as a guard dog. This dog measured approximately 18 inches (46 cm) at the withers, with a lean but muscular frame. The ears have been described both as erect but later described as hanging due to interbreeding with European dogs, but the most distinctive feature was the length of hair often growing in the reverse direction along its back. Within 53 years of the first Dutch settlements in Southern Africa, the Europeans were using these local dogs themselves. By the early 1800s, European colonists had also imported a variety of mainly European dog breeds to this area of Africa, including such dedicated hunting dogs as Greyhounds, Mastiffs, Great Danes and Bloodhounds. The Dogo Cubano (Cuban Bloodhound), an extinct breed used for dogfighting and guarding, was highly emp...	The Border Collie is a breed of British herding dog of the collie type of medium size. It originates in the region of the Anglo-Scottish border, and descends from the traditional sheepdogs once found all over the British Isles. It is kept mostly as a sheep-herding dog or as a companion animal. It competes with success in sheepdog trials. It has been claimed that it is the most intelligent breed of dog.
  Where did the Labrador Retriever originate?	The Labrador breed dates back to at least the 1830s, when St John's water dogs bred by European settlers in Newfoundland were first introduced to Britain from ships trading between Canada and Poole in Dorset. These were then bred with British hunting dogs to create what became known as the Labrador Retriever. Its early patrons included the Earl of Malmesbury, the Duke of Buccleuch, the Earl of Home and Sir John Scott. Early writers have confused the Labrador with the much larger Newfoundland and the Lesser Newfoundland, with Charles William George St John even referring to the Lesser Newfoundland as the Newfoundland. Colonel Peter Hawker describes the first Labrador as being not larger than an English Pointer, more often black than other colours, long in its head and nose with a deep chest, fine legs, and short and smooth coat, and did not carry its tail as highly as the Newfoundland. Hawker distinguishes the Newfoundland from both the "proper Labrador" and St John's breed of these dog...	The Siberian Husky is a breed of medium-sized working sled dog. The breed belongs to the Spitz genetic family. It is recognizable by its thickly furred double coat, erect triangular ears, and distinctive markings, and is smaller than the similar-looking Alaskan Malamute. Siberian Huskies originated in the Russian Far East, where the breed was first developed by the Chukchi people for sled pulling and companionship. Other indigenous Siberian peoples in North Asia, such as the Koryak, Yukaghir, and Kamchadal, also breed them. It is an active, energetic, resilient breed, whose ancestors lived in the extremely cold and harsh environment of the Siberian Arctic. William Goosak, a Russian fur trader, introduced them to Nome, Alaska, during the Nome Gold Rush, initially as sled dogs to work the mining fields and for expeditions through otherwise impassable terrain. Today, the Siberian Husky is typically kept as a house pet, though they are still frequently used as sled dogs by competitive and ...	The name "Brittany" is taken from the Brittany region in northwestern France where the dog originated. Images of orange and white Brittany-like dogs hunting and retrieving game were first seen on tapestries and paintings from the 17th century. The first written and verifiable record of Brittanys comes from a hunting description written by Reverend Davies in 1850. He described hunting with small "bobtailed" dogs who pointed and were excellent retrievers. It was around the same time that the modern Brittany is rumored to have been bred by mating with English Setters. The Brittany was first recognized as a breed in 1907 when an orange and white male named Boy was registered in France. As a result, the first standards were outlined in the same year. America recognized the Brittany in 1931 and the breed was approved by the American Kennel Club in 1934. In 1982 the "Spaniel" was officially dropped from the name.	The Great Dane is a German breed of large mastiff-sighthound, which descends from hunting dogs of the Middle Ages used to hunt bears, wild boar, and deer. They were also used as guardian dogs of German nobility. It is one of the two largest dog breeds in the world, along with the Irish Wolfhound.	The Khoikhoi people who lived in the Cape Peninsula when the Dutch began trading with the area during the mid 17th century, had a hunting dog which was described by Europeans as absolutely fearless and ferocious when acting as a guard dog. This dog measured approximately 18 inches (46 cm) at the withers, with a lean but muscular frame. The ears have been described both as erect but later described as hanging due to interbreeding with European dogs, but the most distinctive feature was the length of hair often growing in the reverse direction along its back. Within 53 years of the first Dutch settlements in Southern Africa, the Europeans were using these local dogs themselves. By the early 1800s, European colonists had also imported a variety of mainly European dog breeds to this area of Africa, including such dedicated hunting dogs as Greyhounds, Mastiffs, Great Danes and Bloodhounds. The Dogo Cubano (Cuban Bloodhound), an extinct breed used for dogfighting and guarding, was highly emp...	The Border Collie is a breed of British herding dog of the collie type of medium size. It originates in the region of the Anglo-Scottish border, and descends from the traditional sheepdogs once found all over the British Isles. It is kept mostly as a sheep-herding dog or as a companion animal. It competes with success in sheepdog trials. It has been claimed that it is the most intelligent breed of dog.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          384,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• gradient_accumulation_steps: 2
• learning_rate: 2e-05
• num_train_epochs: 8
• warmup_steps: 0.1
• bf16: True
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: dorrito-dev/all-MiniLM-L6-v2-dog-breed-retriever
• hub_strategy: end
• batch_sampler: no_duplicates

All Hyperparameters

• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 8
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0.1
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: dorrito-dev/all-MiniLM-L6-v2-dog-breed-retriever
• hub_strategy: end
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

• The bold row denotes the saved checkpoint.

Training Time

• Training: 42.4 seconds

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}