SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("JacobLinCool/Qwen3-Embedding-0.6B-GIR-1")
# Run inference
queries = [
    "Generates samples of text from the provided vocabulary.\n\n  Args:\n    plain_vocab: vocabulary.\n    distribution: distribution.\n    train_samples: samples for training.\n    length: length.\n\n  Returns:\n    train_indices (np.array of Integers): random integers for training.\n      shape = [num_samples, length]\n    test_indices (np.array of Integers): random integers for testing.\n      shape = [num_samples, length]\n    plain_vocab   (list of Integers): unique vocabularies.",
]
documents = [
    'def generate_plaintext_random(plain_vocab, distribution, train_samples,\n                              length):\n  \n  if distribution is not None:\n    assert len(distribution) == len(plain_vocab)\n\n  train_indices = np.random.choice(\n      range(len(plain_vocab)), (train_samples, length), p=distribution)\n\n  return train_indices',
    'def switch(self, name):\n        \n        try:\n            switch = self.storage[self.__namespaced(name)]\n        except KeyError:\n            if not self.autocreate:\n                raise ValueError("No switch named  registered in " % (name, self.namespace))\n\n            switch = self.__create_and_register_disabled_switch(name)\n\n        switch.manager = self\n        return switch',
    'def late_filling(target, pressure=,\n                 Pc_star=,\n                 Swp_star=0.2, eta=3):\n    r\n    element = pressure.split()[0]\n    network = target.project.network\n    phase = target.project.find_phase(target)\n    pc_star = phase[Pc_star]\n    Pc = phase[pressure]\n    \n        Ts = network.map_throats(throats=target.Ts, origin=target)\n        values = values[Ts]\n    else:\n        Ps = network.map_pores(pores=target.Ps, origin=target)\n        values = values[Ps]\n    return values',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.8344, -0.0822,  0.0233]])

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 400 training samples
• Columns: query and code
• Approximate statistics based on the first 400 samples:

  	query	code
  type	string	string
  details	min: 2 tokensmean: 67.12 tokensmax: 3156 tokens	min: 24 tokensmean: 126.98 tokensmax: 1236 tokens

• Samples:

  query	code
  For memory actions, get a list of addresses it operates on. :param SimAction action: The action object to work with. :return: A list of addresses that are accessed with that action. :rtype: list	def _get_actual_addrs(action, state): if action.actual_addrs is None: addr_list = {0x60000000} else: addr_list = set(action.actual_addrs) return addr_list
  Construct the input file of the calculation.	def make_input(self, with_header=False): s = str(self.input) if with_header: s = str(self) + "\n" + s return s
  Check worker status route	def check_worker_status(): if not in request.args: resp = {"status": "bad request"} return jsonify(**resp) else: worker_id = request.args[] assignment_id = request.args[] allow_repeats = CONFIG.getboolean(, ) if allow_repeats: try: part = Participant.query.<br> filter(Participant.workerid == worker_id).<br> filter(Participant.assignmentid == assignment_id).one() status = part.status except exc.SQLAlchemyError: status = NOT_ACCEPTED else: try: matches = Participant.query.<br> filter(Participant.workerid == worker_id).all() numrecs = len(matches) if numrecs==0: status = NOT_ACCEPTED else: status = max([record.status for record in matches]) except exc.SQLAlchemyError: ...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 100 evaluation samples
• Columns: query and code
• Approximate statistics based on the first 100 samples:

  	query	code
  type	string	string
  details	min: 5 tokensmean: 66.56 tokensmax: 548 tokens	min: 24 tokensmean: 142.11 tokensmax: 901 tokens

• Samples:

  query	code
  Return the value of the android prefixed attribute in a specific tag. This function will always try to get the attribute with a android: prefix first, and will try to return the attribute without the prefix, if the attribute could not be found. This is useful for some broken AndroidManifest.xml, where no android namespace is set, but could also indicate malicious activity (i.e. wrongly repackaged files). A warning is printed if the attribute is found without a namespace prefix. If you require to get the exact result you need to query the tag directly: example:: >>> from lxml.etree import Element >>> tag = Element('bar', nsmap={'android': 'http://schemas.android.com/apk/res/android'}) >>> tag.set('{http://schemas.android.com/apk/res/android}foobar', 'barfoo') >>> tag.set('name', 'baz') # Assume that a is some APK object >>> a.get_value_from_tag(tag, 'name'...	def get_value_from_tag(self, tag, attribute): value = tag.get(self._ns(attribute)) if value is None: value = tag.get(attribute) if value: log.warning("Failed to get the attribute on tag with namespace. " "But found the same attribute without namespace!".format(attribute, tag.tag)) return value
  Get information about this object as a dictionary. Used by WebSocket interface to pass some relevant information to client applications.	def get_as_datadict(self): return dict(type=self.class.name, tags=list(self.tags))
  Makes forecast with the estimated model Parameters ---------- h : int (default : 5) How many steps ahead would you like to forecast? past_values : int (default : 20) How many past observations to show on the forecast graph? intervals : Boolean Would you like to show 95% prediction intervals for the forecast? Returns ---------- - Plot of the forecast	def plot_predict(self,h=5,past_values=20,intervals=True,**kwargs): import matplotlib.pyplot as plt import seaborn as sns figsize = kwargs.get(,(10,7)) if self.latent_variables.estimated is False: raise Exception("No latent variables estimated!") else: scale, shape, skewness = self._get_scale_and_shape(self.latent_variables.get_z_values(transformed=True)) previous_value = self.data[-1] forecasted_values = np.ones(h)*self.states[-1] date_index = self.shift_dates(h) simulations = 10000 sim_vector = np.zeros([simulations,h]) t_params = self.transform_z() for n in range(0,simulations): rnd_q = np.random.normal(0,np.sqrt(self.latent_variables.get_z_values(transformed=True)[0]),h) exp = forecasted_values.copy() for t in range(0,h): if t == 0:...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 2025
• bf16: True
• load_best_model_at_end: True
• optim: adamw_torch
• push_to_hub: True
• hub_model_id: JacobLinCool/Qwen3-Embedding-0.6B-GIR-1
• hub_private_repo: False
• gradient_checkpointing: True
• eval_on_start: True
• batch_sampler: no_duplicates

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 2025
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: JacobLinCool/Qwen3-Embedding-0.6B-GIR-1
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 5.1.1
• Transformers: 4.56.2
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.1.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}