SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("training_job_matching_sentence-transformers-paraphrase-multilingual-MiniLM-L12-v2-2024-09-03_13-14-25")
# Run inference
sentences = [
    "Responsable d'élevage en production ovine",
    "de gestion immobilière estimer la valeur d'un bien, d'un produit droit, contentieux et négociationappliquer un cadre juridique ou réglementaire réaliser le suivi des décisions prises en assemblées de copropriété traiter des dossiers de contentieux réaliser la gestion administrative des contrats management animer, coordonner une équipe gestion des ressources humaines gérer les ressources humaines conseil, transmission assurer une médiation constructionétablir l'état d'avancement de travaux piloter la préparation de travaux planifier des travaux de rénovation définir les besoins en rénovation du patrimoine immobilier",
    "de travail et risques professionnelsau domicile d'un particulier déplacements professionnels port d'équipement de protection individuel (epi) : gants, chaussures, casque, protections auditives horaires et durée du travailtravail en astreinte travail le week-end publics spécifiques particuliers secteurs d'activité • bâtiment et travaux publics (btp) 4 / 4 -",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Binary Classification

• Evaluated with BinaryClassificationEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 42,735 training samples
• Columns: name, fiche, and label
• Approximate statistics based on the first 1000 samples:

  	name	fiche	label
  type	string	string	int
  details	min: 3 tokensmean: 9.44 tokensmax: 44 tokens	min: 4 tokensmean: 107.63 tokensmax: 128 tokens	0: ~78.20%1: ~21.80%

• Samples:

  name	fiche	label
  Front End Angular Developer	communication WCF is used. The layer concept enables the reduction of dependencies (dependency injection) of the different tasks (separation of concerns). The entities are exchanged with the database via object-relational mapping (ORM) and processed using the CRUD methods. Through the consistent use of the MVVM pattern, we avoid code-behind. The user interface of the application is realized using the PRISM framework as a "Composite Application UI".Main tasks Software developerIn cooperation with a team located in Germany and respecting the software development guidelines and customers	0
  SCM : Administrateur des ventes	CHEF DE PROJET CONFIRME MAÎTRISANT ANGULAR 4.SON RÔLE SERA L'ENCADREMENT D'UNE EQUIPE ET LA GESTION TOTALE DU DÉVELOPPEMENT D'UNE APPLICATION MOBILE ANDROID.ESPRIT D'ÉQUIPE OBLIGATOIRE.	0
  Talent Acquisition Junior	Pilotage et suivi de toutes les activités du call center (commandes clients et interactions bénéficiaires de la carte).Assurer le calcul et le suivi des Kpi’s du call center.Veiller à la conformité des process et des procédures pour le call center.Contrôle de la prise en charge et la saisie des demandes et réclamations.Pilotage et suivi des projets de la direction clientèle.Assurer toutes demandes ou actions émanant de la Direction Clientèle.Assurer le maintien d’une bonne qualité de service.Augmenter la satisfaction	0

• Loss: SoftmaxLoss

Evaluation Dataset

Unnamed Dataset

• Size: 2,250 evaluation samples
• Columns: name, fiche, and label
• Approximate statistics based on the first 1000 samples:

  	name	fiche	label
  type	string	string	int
  details	min: 3 tokensmean: 9.31 tokensmax: 36 tokens	min: 3 tokensmean: 109.04 tokensmax: 128 tokens	0: ~80.30%1: ~19.70%

• Samples:

  name	fiche	label
  1way com	Nous somme a la recherche de Profils en Telco avec connaissance en Produit! 😃Vous avez une connaissances dans la télécommunication? Emission ou réception (orange, sfr, boygues, free..)Vous voulez travailler dans un environnement stable, accueillant et sans pression?Vous êtes passionnés? Postulez maintenant et profitez d'un salaire motivant et pleins d'avantages:- Salaire 1100 a 1300 (selon le profil)- Primes et challenges- Tickets repas- Transport assuré- Samedi dimanche off- Titularisation- Convention	1
  Senior Front end Web Developer	As part of our growth in Tunis, we are looking to hire a Sénior Front-End Web Developer, who is passionate by Web Development and would like to have a career in an international company, in the Private Banking sector, within an exciting work environment.You will take part, throughout the software development life cycle (SDLC), to the requirement analysis, development and the support of different applications for private banks.You will perform AngularJS frontend development.You will integrate a highly motivated development team working on providing solutions for the private banking sector in which you will integrate the existing global	1
  DÉVELOPPEUR FULLSTACK RUBY ET ANGULAR	professionnel et d'évolution de carrière.- Projets stimulants et variés.- Esprit d'équipe et culture d'entreprise positive.- Salaire compétitif et avantages sociaux attractifs.Rejoignez MCOM et contribuez à révolutionner le commerce mobile avec nous!	0

• Loss: SoftmaxLoss

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 32
• num_train_epochs: 5
• warmup_ratio: 0.1
• bf16: True

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.42.4
• PyTorch: 2.3.1+cu121
• Accelerate: 0.32.1
• Datasets: 2.21.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers and SoftmaxLoss

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}