SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B on the netsecgame-embedding-finetuning-pairs dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • netsecgame-embedding-finetuning-pairs

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("stratosphere/Qwen3-Embedding-0.6B-netsecgame-finetuned-pairs")
# Run inference
queries = [
    "Current status:\nControlled hosts are 10.7.44.3 and 10.7.44.5 and 10.7.44.4 and 54.123.53.21 and 10.7.44.2\nKnown networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24\nKnown hosts are 10.7.45.4 and 10.7.45.3 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2\nKnown services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and \nKnown services for host 10.7.44.5 are 22/tcp, ssh and bash and \nKnown services for host 54.123.53.21 are listener and bash and \nKnown services for host 10.7.44.4 are 22/tcp, ssh and bash and \nKnown services for host 10.7.44.3 are 3389/tcp, ms-wbt-server and \nKnown services for host 10.7.45.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and \nKnown data are none\n",
]
documents = [
    'Current status:\nControlled hosts are 172.19.2.2 and 101.32.5.23 and 172.19.1.4 and 172.19.1.3 and 172.19.1.2\nKnown networks are 54.123.53.29/26 and 172.19.0.0/24 and 172.19.1.0/24 and 172.19.2.0/24\nKnown hosts are 172.19.2.4 and 172.19.2.3 and 172.19.1.5 and 172.19.1.1 and 172.19.1.6\nKnown services for host 172.19.1.4 are bash and 22/tcp, ssh and \nKnown services for host 172.19.1.3 are 3389/tcp, ms-wbt-server and \nKnown services for host 101.32.5.23 are bash and listener and \nKnown services for host 172.19.1.5 are 22/tcp, ssh and \nKnown services for host 172.19.2.3 are 22/tcp, ssh and 5432/tcp, postgresql and \nKnown services for host 172.19.1.2 are 3389/tcp, ms-wbt-server and \nKnown services for host 172.19.2.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and \nKnown data are none\n',
    'Current status:\nControlled hosts are 172.20.7.6 and 32.8.23.1\nKnown networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26\nKnown hosts are 172.20.7.2 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4\nKnown services for host 172.20.8.2 are 445/tcp, microsoft-ds and 3389/tcp, ms-wbt-server and \nKnown services for host 172.20.8.4 are 22/tcp, ssh and 80/tcp, http and \nKnown services for host 172.20.7.6 are bash and \nKnown data are none\n',
    'Current status:\nControlled hosts are 172.20.8.3 and 172.20.7.2 and 32.8.23.1 and 172.20.8.4\nKnown networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26\nKnown hosts are 172.20.7.6 and 172.20.7.4 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5\nKnown services for host 32.8.23.1 are listener and bash and \nKnown services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and \nKnown services for host 172.20.8.4 are 22/tcp, ssh and bash and 80/tcp, http and \nKnown services for host 172.20.8.3 are 22/tcp, ssh and 5432/tcp, postgresql and \nKnown data for host 172.20.8.4 are (User2, WebServerData) and \n',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5508, 0.4785, 0.4883]], dtype=torch.bfloat16)

Evaluation

Metrics

Binary Classification

• Dataset: topology_val
• Evaluated with BinaryClassificationEvaluator

Training Details

Training Dataset

netsecgame-embedding-finetuning-pairs

• Dataset: netsecgame-embedding-finetuning-pairs at 1a27835
• Size: 21,011 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	float
  details	min: 111 tokensmean: 338.8 tokensmax: 577 tokens	min: 111 tokensmean: 339.66 tokensmax: 658 tokens	min: 0.0mean: 0.5max: 1.0

• Samples:

  sentence1	sentence2	label
  Current status:Controlled hosts are 32.8.23.1 and 172.20.7.3Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.2 are 3389/tcp, ms-wbt-server and Known services for host 172.20.7.5 are 22/tcp, ssh and Known data are none	Current status:Controlled hosts are 32.8.23.1 and 172.20.7.5Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.8.4Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 172.20.7.4 are 22/tcp, ssh and Known data are none	1.0
  Current status:Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2 and 192.168.2.4Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24Known hosts are 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.2 are 3389/tcp, ms-wbt-server and Known data are none	Current status:Controlled hosts are 10.7.45.4 and 10.7.44.2 and 10.7.44.4 and 54.123.53.21Known networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24Known hosts are 10.7.44.3 and 10.7.45.3 and 10.7.44.5 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2Known services for host 54.123.53.21 are listener and bash and Known services for host 10.7.44.4 are 22/tcp, ssh and Known services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 10.7.45.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0
  Current status:Controlled hosts are 172.20.7.2 and 32.8.23.1Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26Known hosts are 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4Known services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known data are none	Current status:Controlled hosts are 32.8.23.1 and 172.20.7.3Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4Known services for host 172.20.7.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.2 are 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known data are none	1.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Evaluation Dataset

netsecgame-embedding-finetuning-pairs

• Dataset: netsecgame-embedding-finetuning-pairs at 1a27835
• Size: 2,626 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	float
  details	min: 147 tokensmean: 338.35 tokensmax: 567 tokens	min: 147 tokensmean: 339.84 tokensmax: 599 tokens	min: 0.0mean: 0.51max: 1.0

• Samples:

  sentence1	sentence2	label
  Current status:Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24Known hosts are 192.168.2.4 and 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2Known services for host 192.168.2.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.3 are 3389/tcp, ms-wbt-server and Known data are none	Current status:Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24Known hosts are 192.168.2.4 and 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2Known services for host 192.168.2.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.2 are 3389/tcp, ms-wbt-server and Known data are none	1.0
  Current status:Controlled hosts are 10.7.45.4 and 10.7.44.2 and 10.7.44.4 and 54.123.53.21Known networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24Known hosts are 10.7.44.3 and 10.7.45.3 and 10.7.44.5 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2Known services for host 54.123.53.21 are listener and bash and Known services for host 10.7.44.4 are 22/tcp, ssh and Known services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 10.7.45.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	Current status:Controlled hosts are 172.19.1.2 and 101.32.5.23Known networks are 54.123.53.29/26 and 172.19.0.0/24 and 172.19.1.0/24 and 172.19.2.0/24Known hosts are 172.19.2.2 and 172.19.1.4 and 172.19.2.4 and 172.19.2.3 and 172.19.1.5 and 172.19.1.1 and 172.19.1.3 and 172.19.1.6Known services for host 101.32.5.23 are bash and listener and Known services for host 172.19.1.4 are 22/tcp, ssh and Known services for host 172.19.1.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.19.2.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0
  Current status:Controlled hosts are 172.20.7.4 and 172.20.7.2 and 32.8.23.1 and 172.20.8.2Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26Known hosts are 172.20.7.6 and 172.20.8.3 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4Known services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.4 are 22/tcp, ssh and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 172.20.8.2 are 445/tcp, microsoft-ds and 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.5 are 22/tcp, ssh and Known services for host 172.20.8.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	Current status:Controlled hosts are 10.0.47.2 and 10.0.46.6 and 55.34.2.5 and 10.0.47.4Known networks are 10.0.46.0/24 and 10.0.45.0/24 and 55.34.2.4/26 and 10.0.47.0/24Known hosts are 10.0.46.1 and 10.0.47.3 and 10.0.46.2 and 10.0.46.5 and 10.0.46.4 and 10.0.46.3Known services for host 55.34.2.5 are bash and listener and Known services for host 10.0.46.5 are 22/tcp, ssh and Known services for host 10.0.47.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and Known services for host 10.0.46.6 are bash and Known services for host 10.0.47.3 are 22/tcp, ssh and 5432/tcp, postgresql and Known services for host 10.0.46.3 are 3389/tcp, ms-wbt-server and Known services for host 10.0.47.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• gradient_accumulation_steps: 4
• learning_rate: 1e-06
• weight_decay: 0.01
• num_train_epochs: 1
• load_best_model_at_end: True

All Hyperparameters

• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-06
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.2.2
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}