SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'MPNetModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Unity Development. Game Development. Mobile Game Development. Game Design. Game Art. 2D Game Art. 3D Game Art. 3D Modeling. Level Design. Game Prototype. Game UI/UX Design. Augmented Reality. Virtual Reality. Animation. VFX. Game Documentation. Game Balance. Analytics Integration. Monetization Strategy. User Acquisition. Unity. Unity Game Development. Creates full-cycle 2D and 3D mobile games from concept to launch and support.. Game Art and UI/UX. Designs 2D/3D art assets, animations, VFX, and user interfaces for games.. Game Design and Monetization. Builds GDDs, wireframes, balance systems, analytics integration, and monetization features.. Senior Game Development Expert | Mind Studios. Works on full-cycle development of 2D and 3D mobile games, from idea to launch and support (September 2016 - Present).',
    'Machine Learning Engineer\n\n        Requirements:\n        Python, Machine Learning, Deep Learning,\n        NLP, TensorFlow, PyTorch, SQL,\n        Data Science, Artificial Intelligence,\n        Generative AI, Transformers.',
    'Machine Learning Engineer\n\n        Requirements:\n        Python, Machine Learning, Deep Learning,\n        NLP, TensorFlow, PyTorch, SQL,\n        Data Science, Artificial Intelligence,\n        Generative AI, Transformers.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.0610, 0.0610],
#         [0.0610, 1.0000, 1.0000],
#         [0.0610, 1.0000, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,953 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  modality	text	text
  details	min: 91 tokensmean: 269.21 tokensmax: 384 tokens	min: 34 tokensmean: 39.33 tokensmax: 42 tokens	min: 0.04mean: 0.21max: 0.82

• Samples:

  sentence_0	sentence_1	label
  Program management. Cybersecurity. Risk assessment. Vulnerability management. Business continuity. Standardization. Policy and coordination. Web security. Adversary emulation. Adversary emulation services. Enterprise CIS security frameworks. Business continuity planning tools. Web Presence Security Program. Managed a NATO web presence security program to improve cybersecurity posture and implement adversary emulation services.. Enterprise CIS Security Initiatives. Led enterprise-wide CIS security coherence and vulnerability management activities to strengthen NATO security and resilience.. NATO Business Continuity Project. Coordinated business continuity activities to support NATO operational resilience during transitions.. Program Manager - NATO OCIO. Manage cybersecurity programs focused on ICT coherence, enterprise CIS security, vulnerability management, and adversary emulation (March 2022 - Present).. Business Continuity Project Coordinator - NATO. Coordinated business continuity p...	Cybersecurity Engineer Requirements: Network Security, Ethical Hacking, Penetration Testing, Vulnerability Assessment, Python, Linux, Wireshark, Nmap, SIEM, Incident Response, Digital Forensics.	0.7298
  Cloud Platform Engineering. DevOps. Site Reliability Engineering. Infrastructure as Code. CI/CD. Kubernetes. Docker. Cloud Migration. Observability. Security and Compliance. Terraform. Platform Architecture. Database / Data Platform. EKS. ECS. Lambda. VPC. IAM. RDS. S3. CloudWatch. CloudTrail. GuardDuty. API Gateway. AKS. Data Factory. Functions. Key Vault. Entra ID. Private Link. Azure Monitor. Terraform. Helm. Kustomize. Ansible. ArgoCD. Flux. GitHub Actions. GitLab CI. Azure DevOps. Jenkins. Prometheus. Grafana. ELK. Loki. OpenTelemetry. PagerDuty. Vault. OPA. Kyverno. Trivy. Snowflake. ADLS Gen2. Power BI. Python. Bash. YAML. HCL. On-prem to AWS Migration. Led phased zero-downtime migration from on-prem to AWS with multi-AZ EKS clusters, segmented VPCs, and modular Terraform reducing infrastructure drift by ~90%.. Cloud-native Platform with GitOps. Designed cloud-native platform using Kubernetes, ArgoCD, hardened CI/CD with image scanning and SAST, full observability, Vault secrets...	Machine Learning Engineer Requirements: Python, Machine Learning, Deep Learning, NLP, TensorFlow, PyTorch, SQL, Data Science, Artificial Intelligence, Generative AI, Transformers.	0.1524
  Cyber resilience. Information security management. ISO 27001. Risk management. Security assessments. Gap analysis. CISO-as-a-Service. Project management. Customer success. Training and awareness. ISO/IEC 27001. NIST CSF. OneTrust (inferred from GRC tooling experience). M365. BSI IT-Grundschutz. ISMS Implementation and Audit. Led ISMS build, audits, security assessments and gap analyses aligned with ISO/IEC 27001 and NIST CSF to improve organisational security posture.. Regulatory Readiness for NIS2 & CRA. Advised clients on compliance and translated regulatory requirements (NIS2, Cyber Resilience Act, FINMA) into practical measures and governance.. M365 Rollout and OKR Adoption. Managed M365 rollout projects and introduced OKR agile management practices while coordinating project risks and stakeholder engagement.. Cyber Security Consultant - InfoGuard AG. Consulting on cyber security, ISMS implementation, audits, risk analyses and regulatory compliance including NIS2 and Cyber Resilien...	Cloud Engineer Requirements: AWS, Azure, Google Cloud, Docker, Kubernetes, Terraform, Linux, DevOps, CI/CD, Cloud Infrastructure, Cloud Security.	0.16920000000000002

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss",
      "cos_score_transformation": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• per_device_train_batch_size: 16
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 16
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 13.4 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.13.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}