SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Instruct: \nQuery:  It is branded as Dynorm, Inhibace, Vascace and many other names in various countries. None of these are available in the United States as of May 2010.",
]
documents = [
    'Instruct: \nQuery:  Enantiopure drugs',
    'Instruct: \nQuery:  Bioluminescence',
    'Instruct: \nQuery:  Electrochemists',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4157, 0.1706, 0.1681]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,976 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 10 tokensmean: 228.29 tokensmax: 1024 tokens	min: 10 tokensmean: 28.26 tokensmax: 595 tokens

• Samples:

  anchor	positive
  Instruct: Query: Control Measures
  *
  *
  Contractor Summary
  *
  *
  Ingredients
  *
  % Wt: <0.1
  OSHA PEL: C 0.1 MG(CRO3)/M3
  ACGIH TLV: 0.5 MG(CR)/M3, A4
  ------------------------------
  OSHA PEL: N/K (FP N)
  ACGIH TLV: N/K (FP N)
  ------------------------------
  OSHA PEL: N/K (FP N)
  ACGIH TLV: N/K (FP N)
  -----------------------------
  GUIDELINES ARE BASED ON FEDERAL REGS &
  -----------------------------
  PLEASE CONSULT LOCAL ENVIRONMENTAL
  -----------------------------
  *
  Health Hazards Data
  *
  Route Of Entry Inds - Inhalation: YES
  Skin: YES
  Ingestion: YES
  Carcinogenicity Inds - NTP: YES
  IARC: YES
  OSHA: NO
  Effects of Exposure: ACUTE: EYE CONTACT, SKIN CONTACT, SKIN ABSORP: NO EFFECTS
  ANTICIPATED. INGESTION: PRACTICALLY NON-TOXIC. INHALATION: NO EFFECTS
  ANTICIPATED. POTASSIUM DICHROMATE: CARCINOGEN. EXPERMENTAL MUTAGEN. EXPE
  RIMENTAL TERATOGEN. CAUSES BURNS. OXIDIZER. TOXIC. MAY CAUSE ALLERGIC
  REACTION. CHRONIC EFFECTS: NO EFTS ANTICIPATED.
  Signs And S...	Instruct: Query: eyes_protection_mandatory
  Instruct: Query: * Exposure Controls/Personal Protection Respiratory Protection:FOLLOW THE OSHA RESPIRATOR REGULATIONS FOUND IN NECESSARY.Ventilation:USE ADEQUATE GENERAL OR LOCAL EXHAUST VENTILATION TO KEEP AIRBORNE CONCENTRATIONS BELOW THE PERMISSIBLE EXPOSURE LIMITS.Other Protective Equipment:ANSI APPROVED EMERGENCY EYEWASH AND DELUGE SHOWER . WEAR APPROPRIATE PROTECTIVE CLOTHING TO PREVENT SKIN EXPOSURE.Work Hygienic Practices:WASH THOROUGHLY AFTER HANDLING. REMOVE CONTAMINATED CLOTHING AND WASH BEFORE REUSE.Supplemental Safety and HealthMATLS TO AVOID: MATERIALS AND COMBUSTIBLES. FIRST AID PROC: IMMED. IF NOT BRTHG, GIVE ARTF RESP. IF BRTHG IS DFCLT, GIVE OXYGEN. GET MED AID. NOTES TO MD: TREAT SYMPTOMATICALLY & SUPPORTIVELY. NO SPEC IFIC ANTIDOTE EXISTS. Product Identification Composition/Information on Ingredients OSHA PEL:2 PPMACGIH TLV:2 PPM/4 STELOSHA PEL:N/K ACGIH TLV:N/K Hazards Identification *Routes of Entry:...	Instruct: Query: gloves_mandatory
  Instruct: Query: The isomerization of uridine to pseudoridine is the second most common rRNA modification. These pseudoridines are also introduced by the same classes of snoRNPs that participate in methylation. Psuedouridine synthases are the major participating enzymes in the reaction. The H/ACA box snoRNPs introduce guide sequences that are about 14-15 nucleotides long. Pseudouridylation is triggered in numerous places of rRNAs at once to preserve the thermal stability of RNA. Pseudouridine allows for increased hydrogen bonding and alters translation in rRNA and tRNA. It alters translation by increasing the affinity of the ribosome subunit to specific mRNAs.Base Editing:Base editing is the third major class of rRNA modification, specifically in eukaryotes. There are 8 categories of base edits that can occur at the gap between the small and large ribosomal subunits. RNA methyltransferases are the enzymes that introduce base methylation. Acetyltransferases are the enzymes responsib...	Instruct: Query: Gene expression + Signal Transduction

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• learning_rate: 0.0001
• num_train_epochs: 1
• bf16: True

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0001
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}