SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("along26/distilbert-base-manglish-sentence-transformer")
# Run inference
sentences = [
    'To find the shortest distance, we need to find the shortest path that visits all cities exactly once. We can use the brute force method to find all possible routes and then choose the shortest one.\n\nThere are 4! (factorial) possible routes since the salesman can start from City 1 and then visit the other cities in any order. 4! = 4 × 3 × 2 × 1 = 24 possible routes.\n\nHere are all the possible routes and their distances:\n\n1. 1-2-3-4-5: 10 + 8 + 14 + 16 = 48 km\n2. 1-2-3-5-4: 10 + 8 + 28 + 16 = 62 km\n3. 1-2-4-3-5: 10 + 25 + 14 + 28 = 77 km\n4. 1-2-4-5-3: 10 + 25 + 16 + 28 = 79 km\n5. 1-2-5-3-4: 10 + 18 + 14 + 14 = 56 km\n6. 1-2-5-4-3: 10 + 18 + 16 + 14 = 58 km\n7. 1-3-2-4-5: 15 + 8 + 25 + 16 = 64 km\n8. 1-3-2-5-4: 15 + 8 + 18 + 16 = 57 km\n9. 1-3-4-2-5: 15 + 14 + 25 + 18 = 72 km\n10. 1-3-4-5-2: 15 + 14 + 16 + 18 = 63 km\n11. 1-3-5-2-4: 15 + 28 + 18 + 25 = 86 km\n12. 1-3-5-4-2: 15 + 28 + 16 + 25 = 84 km\n13. 1-4-2-3-5: 12 + 8 + 14 + 28 = 62 km\n14. 1-4-2-5-3: 12 + 8 + 18 + 28 = 66 km\n15. 1-4-3-2-5: 12 + 14 + 8 + 18 = 52 km\n16. 1-4-3-5-2: 12 + 14 + 28 + 18 = 72 km\n17. 1-4-5-2-3: 12 + 16 + 18 + 8 = 54 km\n18. 1-4-5-3-2: 12 + 16 + 28 + 8 = 64 km\n19. 1-5-2-3-4: 20 + 18 + 14 + 14 = 66 km\n20. 1-5-2-4-3: 20 + 18 + 25 + 14 = 77 km\n21. 1-5-3-2-4: 20 + 28 + 8 + 25 = 81 km\n22. 1-5-3-4-2: 20 + 28 + 14 + 25 = 87 km\n23. 1-5-4-2-3: 20 + 16 + 25 + 8 = 69 km\n24. 1-5-4-3-2: 20 + 16 + 14 + 8 = 58 km\n\nThe shortest distance is 48 km, and the route is 1-2-3-4-5.',
    'Wah, to find shortest distance ah, we need find shortest path that visit all city exactly once lah. Can use brute force method to find all possible route and then choose the shortest one lor.\n\nGot 4! (factorial) possible route since salesman can start from City 1 and then visit other city in any order. 4! = 4 × 3 × 2 × 1 = 24 possible route lah.\n\nHere all the possible route and their distance:\n\n1. 1-2-3-4-5: 10 + 8 + 14 + 16 = 48 km\n2. 1-2-3-5-4: 10 + 8 + 28 + 16 = 62 km\n3. 1-2-4-3-5: 10 + 25 + 14 + 28 = 77 km\n4. 1-2-4-5-3: 10 + 25 + 16 + 28 = 79 km\n5. 1-2-5-3-4: 10 + 18 + 14 + 14 = 56 km\n6. 1-2-5-4-3: 10 + 18 + 16 + 14 = 58 km\n7. 1-3-2-4-5: 15 + 8 + 25 + 16 = 64 km\n8. 1-3-2-5-4: 15 + 8 + 18 + 16 = 57 km\n9. 1-3-4-2-5: 15 + 14 + 25 + 18 = 72 km\n10. 1-3-4-5-2: 15 + 14 + 16 + 18 = 63 km\n11. 1-3-5-2-4: 15 + 28 + 18 + 25 = 86 km\n12. 1-3-5-4-2: 15 + 28 + 16 + 25 = 84 km\n13. 1-4-2-3-5: 12 + 8 + 14 + 28 = 62 km\n14. 1-4-2-5-3: 12 + 8 + 18 + 28 = 66 km\n15. 1-4-3-2-5: 12 + 14 + 8 + 18 = 52 km\n16. 1-4-3-5-2: 12 + 14 + 28 + 18 = 72 km\n17. 1-4-5-2-3: 12 + 16 + 18 + 8 = 54 km\n18. 1-4-5-3-2: 12 + 16 + 28 + 8 = 64 km\n19. 1-5-2-3-4: 20 + 18 + 14 + 14 = 66 km\n20. 1-5-2-4-3: 20 + 18 + 25 + 14 = 77 km\n21. 1-5-3-2-4: 20 + 28 + 8 + 25 = 81 km\n22. 1-5-3-4-2: 20 + 28 + 14 + 25 = 87 km\n23. 1-5-4-2-3: 20 + 16 + 25 + 8 = 69 km\n24. 1-5-4-3-2: 20 + 16 + 14 + 8 = 58 km\n\nThe shortest distance is 48 km, and the route is 1-2-3-4-5 lor.',
    "Najib Razak's corruption trial has had a significant impact on Malaysia's political landscape and its fight against corruption. Najib, who served as Malaysia's Prime Minister from 2009 to 2018, has been on trial since 2019 on multiple charges of corruption, money laundering, and abuse of power related to the 1Malaysia Development Berhad (1MDB) scandal.\n\nThe 1MDB scandal involved the misappropriation of billions of dollars from a Malaysian state investment fund, and it has been described as one of the world's largest cases of financial misconduct. The scandal played a significant role in the ousting of Najib's ruling coalition, the Barisan Nasional, in the 2018 general election, which was won by the opposition alliance led by Mahathir Mohamad.\n\nNajib's corruption trial has been closely watched both domestically and internationally, as it represents a high-profile test of Malaysia's commitment to tackling corruption and upholding the rule of law. The trial has helped to raise public awareness of the issue of corruption in Malaysia and has put pressure on the government to take action against corrupt officials.\n\nThe trial has also had a significant impact on Malaysia's political landscape. Najib's United Malays National Organization (UMNO) party, which was once the dominant political force in the country, has been weakened by the scandal and the trial. The party was forced to leave the ruling coalition after Najib's election defeat, and it has struggled to regain its former influence.\n\nAt the same time, the trial has bolstered the credibility of the Malaysian Anti-Corruption Commission (MACC) and other anti-corruption agencies, which have been empowered to take on high-level cases of corruption. The MACC has reported an increase in the number of corruption cases reported since Najib's election defeat and the launch of his trial.\n\nIn summary, Najib Razak's corruption trial has been a significant milestone in Malaysia's fight against corruption, raising public awareness of the issue and bolstering the credibility of anti-corruption agencies. The trial has also had a profound impact on Malaysia's political landscape, weakening the influence of UMNO and highlighting the need for greater transparency and accountability in government.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -1.0000,  1.0000],
#         [-1.0000,  1.0000, -1.0000],
#         [ 1.0000, -1.0000,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 139,404 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 13 tokensmean: 145.81 tokensmax: 256 tokens	min: 16 tokensmean: 165.11 tokensmax: 256 tokens	min: 12 tokensmean: 144.87 tokensmax: 256 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  The concentration of reactants plays a significant role in determining the rate and efficiency of a chemical reaction. According to the collision theory, a chemical reaction occurs when reactant particles collide with each other with sufficient energy and proper orientation. The rate of a chemical reaction depends on the frequency and effectiveness of these collisions.Here's how the concentration of reactants affects the rate and efficiency of a chemical reaction:1. Increased concentration of reactants: When the concentration of reactants is increased, there are more reactant particles per unit volume. This leads to a higher probability of collisions between reactant particles, resulting in an increased rate of reaction. In general, the rate of a reaction is directly proportional to the concentration of the reactants, as described by the rate law equation:Rate = k[A]^m[B]^nwhere Rate is the reaction rate, k is the rate constant, [A] and [B] are the concentrations of reactants A...	Kepekatan bahan tindak balas memainkan peranan penting dalam menentukan kadar dan kecekapan tindak balas kimia. Menurut teori perlanggaran, tindak balas kimia berlaku apabila zarah bahan tindak balas berlanggar antara satu sama lain dengan tenaga yang mencukupi dan orientasi yang betul. Kadar tindak balas kimia bergantung kepada kekerapan dan keberkesanan perlanggaran ini.Begini cara kepekatan bahan tindak balas mempengaruhi kadar dan kecekapan tindak balas kimia:1. Peningkatan kepekatan bahan tindak balas: Apabila kepekatan bahan tindak balas meningkat, terdapat lebih banyak zarah bahan tindak balas per unit isipadu. Ini membawa kepada kebarangkalian perlanggaran yang lebih tinggi antara zarah bahan tindak balas, mengakibatkan peningkatan kadar tindak balas. Secara umum, kadar tindak balas adalah berkadar terus dengan kepekatan bahan tindak balas, seperti yang diterangkan oleh persamaan undang-undang kadar:Kadar = k[A]^m[B]^ndi mana Kadar ialah kadar tindak balas, k ialah pema...	Najib Razak, who served as the Prime Minister of Malaysia from 2009 to 2018, was known for his pro-business and pro-investment approach to foreign policy, which included strengthening Malaysia's ties with China. However, this approach also contributed to criticism from the left in Malaysia for several reasons.Firstly, critics argued that Najib's emphasis on economic ties with China led to an over-reliance on Chinese investment, which could compromise Malaysia's sovereignty and independence. This concern was amplified by the fact that many of the Chinese-funded projects in Malaysia were mega-infrastructure projects, such as the East Coast Rail Link (ECRL) and the Bandar Malaysia development, which were financed through Chinese loans. Critics feared that Malaysia could be burdened with unsustainable debt levels, which could have long-term implications for the country's economic stability.Secondly, the left in Malaysia raised concerns about the environmental impact of some of the Chin...
  To compute the Lie derivative of $X$ with respect to $Y$, we use the Lie bracket of vector fields, denoted by $[Y, X]$. The Lie bracket is defined as $[Y, X] = YX - XY$, where $YX$ and $XY$ denote the action of the vector fields $Y$ and $X$ on functions.Let $f(x, y)$ and $g(x, y)$ be smooth functions on $M$. Then, we have$X = (x^2 + 2y^2) \partial_x + (x - 2y) \partial_y$ and $Y = \partial_x + \partial_y$.Now, we compute the Lie bracket $[Y, X]$:$YX = Y((x^2 + 2y^2) \partial_x + (x - 2y) \partial_y) = Y((x^2 + 2y^2) \partial_x) + Y((x - 2y) \partial_y)$.Using the product rule, we get$YX = ((2x) \partial_x + (4y) \partial_y)(x^2 + 2y^2) \partial_x + ((1) \partial_x + (-2) \partial_y)(x - 2y) \partial_y$.Now, we compute $XY$:$XY = X(\partial_x + \partial_y) = X(\partial_x) + X(\partial_y)$.Using the product rule, we get$XY = ((x^2 + 2y^2) \partial_x + (x - 2y) \partial_y) \partial_x + ((x^2 + 2y^2) \partial_x + (x - 2y) \partial_y) \partial_y$.Now, we compute the Li...	Untuk mengira terbitan Lie bagi $X$ berkenaan dengan $Y$, kami menggunakan kurungan Lie bagi medan vektor, dilambangkan dengan $[Y, X]$. Kurungan Lie ditakrifkan sebagai $[Y, X] = YX - XY$, di mana $YX$ dan $XY$ menandakan tindakan medan vektor $Y$ dan $X$ pada fungsi.Biarkan $f(x, y)$ dan $g(x, y)$ menjadi fungsi lancar pada $M$. Kemudian, kita ada$X = (x^2 + 2y^2) \partial_x + (x - 2y) \partial_y$ dan $Y = \partial_x + \partial_y$.Sekarang, kita mengira kurungan Lie $[Y, X]$:$YX = Y((x^2 + 2y^2) \partial_x + (x - 2y) \partial_y) = Y((x^2 + 2y^2) \partial_x) + Y((x - 2y) \partial_y )$.Menggunakan peraturan produk, kita dapat$YX = ((2x) \partial_x + (4y) \partial_y)(x^2 + 2y^2) \partial_x + ((1) \partial_x + (-2) \partial_y)(x - 2y) \partial_y$ .Sekarang, kita mengira $XY$:$XY = X(\partial_x + \partial_y) = X(\partial_x) + X(\partial_y)$.Menggunakan peraturan produk, kita dapat$XY = ((x^2 + 2y^2) \partial_x + (x - 2y) \partial_y) \partial_x + ((x^2 + 2y^2) \partial_...	Ensuring fairness and impartiality in trials and appeals related to high-level corruption cases like the 1MDB scandal requires effective monitoring and oversight mechanisms to detect and address potential political interference. Here are some specific mechanisms that can be put in place, along with the entities responsible for implementing and overseeing them:1. Judicial Commission: Establishing an independent judicial commission can provide oversight of the judiciary and investigate allegations of political interference. The commission could be responsible for reviewing judicial appointments, promotions, and disciplinary actions, and for recommending reforms to promote judicial independence. This commission should be composed of respected legal experts, civil society representatives, and other stakeholders, and should be independent from the government and political influence.2. Anti-corruption agency: A dedicated anti-corruption agency can investigate allegations of corruption and...
  What is the reaction mechanism for the acid-catalyzed cleavage of an ether? Provide the step-by-step process and the structures of the intermediate compounds involved in the reaction.	Apakah mekanisme tindak balas bagi pembelahan bermangkin asid bagi eter? Sediakan proses langkah demi langkah dan struktur sebatian perantaraan yang terlibat dalam tindak balas.	Political funding in Malaysia has been criticized for being opaque and vulnerable to corruption due to several reasons. Firstly, there is a lack of transparency in the way political parties raise and spend their funds. Many political donations are made in cash, and there is no requirement for parties to disclose the source of their donations. This lack of transparency creates an environment that is conducive to corruption, as it allows politicians to accept illicit funds without being held accountable.Secondly, there are no limits on the amount of money that political parties can receive from donors. This has led to concerns that wealthy individuals or corporations may use their financial influence to gain favorable policies or access to decision-makers. In some cases, political donations have been linked to corrupt practices such as cronyism and patronage.Thirdly, the enforcement of laws related to political funding is weak. The Malaysian Anti-Corruption Commission (MACC) has limi...

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}