SentenceTransformer based on Qwen/Qwen3-Embedding-4B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-4B on the json dataset. It maps sentences & paragraphs to a 2560-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-4B
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 2560 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen3Model 
  (1): Pooling({'word_embedding_dimension': 2560, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'w many? Couple thousand?" Naturally, Swift\'s hope had been in vain. He\'d objected to being sent out with someone so green, but orders were orders. There had been reassurances that Private Gallant Heart had showed excellent potential during training, that the mission wasn\'t dangerous, that Swift was more than capable of handling the both of them. Swift lacked his superiors\' certainty. Every mission was dangerous. Scrutinizing the crawling sea of troops, the captain counted their banners. After a few minutes, he\'d picked out four. Though he couldn\'t see the details, he recognised the colour schemes. A dark green flag for a regiment of engineers, two red banners for infantry and one of the pale blue ones for the crossbow dogs. Four banners. Four regiments. Each regiment about 600 troops strong. Each 600 troops having 8 or 9 officers. Plus the cooks, quartermasters, keepers of the baggage train... "Between two and a half and three thousand," Swift stated, keeping his voice low, speaking softly. "You think so, captain?" Gallant asked, his voice barely below conversation level. Why would he say that if he didn\'t think it? Swift held his tongue though, though his partner continued. "What do you think they\'re travelling up here for, captain?" His partner\'s speech made Swift wince. Already, Gallant was showing his lack of experience. Talking so openly? Addressing him by his rank during a mission? All it would take is one hostile snooping around and their cover would be compromised. Of course, if they were captured, Gallant\'s parents would ransom him back before anyone even noticed Swift was missing. The perks of having a rich, well connected family Swift supposed. "Silence." Swift frowned beneath his cloak. He knew that in time, Gallant would be moved to a different branch of the military. He wasn\'t suited for the Equestrian Expeditionary Force, the EED. Heck, it was right there in his name: Gallant Heart. Brave. Courageous. Heroic. Fine things for a knight in an old fairy t',
    'n the dark clearing. There was a soft thud as Swift\'s throwing knife slammed into her neck, just below her skull. She seized up, and fell, clawing desperately at her throat. Swift leap, aiming for the screeching male griffon. He slammed into the scout, knocking him over and winding him. Quickly standing, the pegasus slammed his front hooves onto the griffon\'s skull. Putting his whole weight into it, the first blow knocked the griffon senseless. The second gave a sickening crack as the griffon\'s skill began to give. The third and final strike gave a disgusting squelching noise, dark globs spattering the pony. The griffon was finally silent, his skull reduced to mush. Swift stood and drew his own knife. The other griffon had managed to pull out the throwing knife, her claws desperately clamped over the wound, a strange wheezing noise the only sound she could make. With a few steps, Swift was close enough to finish her. His knife slid into the base of her skull in an easy motion born of countless repetitions, putting an end to her. The whole encounter had taken less than 5 seconds. In that time, Gallant hadn\'t moved an inch, his mouth hanging open, his hoof outstretched like he was still holding his knife. Swift moved back to his shocked partner and gave him a strong slap. "Focus, private." The rookie nodded, blinking like a moron. Swift would have liked to give him a good kicking right about then, but there were bigger things to worry about. "On me Gallant. Like glue." If the alarm was raised, they would be outflown by the griffons. Over flat terrain, at lower altitudes and without any strong wind, a pegasus\' smaller wingspan allowed for quicker manoeuvring, for higher acceleration. Griffons were, on the whole, built for endurance: they could fly much further than a pegasus, their larger wings allowing them to fly higher, to use the winds more efficiently. The pair hustled through the copse, silent trees surrounding them. Every bush a potential enemy, every waving bra',
    'oldiers; Magnum broke it with his own curiosity. "Now you tell me. All about yourself, that is... Sir," he quickly added. Too many times had he forgotten that was he addressing a superior officer and not just a friend. "Heh," the stallion snorted, recollecting the memories from the chasms of his mind. "About me, eh? Alright, I\'ll humor you. If only to pass the time until we reach the drop-point." "I began my military career as Private Third Class \'Gilded Armour\' in the Canterlot Guard, though my friends all called me ‘Gil’. Back then, a civil war like this was the farthest thing from anypony\'s mind, due to the Griffon tribes banding under one Warlord and setting their sights on Equestria. For ten years I was sent all over the place, working my way up through the ranks." Magnum sat, listening intently. He motioned for the story-teller to continue the tale. "Well, it was during a routine visit to Canterlot for a resupply and debrief that the Griffons decided raiding small, defenseless villages wasn\'t enough. It seemed like they sent their entire species to invade Canterlot, there were so many. They outnumbered our Pegasus Air Force three-to-one. Our only advantage was their bulk; Pegasi can out-fly a cumbersome Griffon any day." Armour took a quick breath, letting the memories flood back. "Unfortunately, it meant shit-all. They still backed us up into the Castle proper. Of course, we hardly had this technology at all; it was all spears and other melee weaponry. We were pushed back into the Royal Throne Room; hell I\'m pretty sure my flank touched Celestia, praise her name. The only saving grace that day was when the Princesses decided to join the fight... Princess Luna herself was particularly nasty." "Eventually they had been routed after sustaining a heavy loss... but I celebrated too early. One last of those bird-lion mutts flew at Celestia herself in a blind fury. She stood frozen, in shock at the audacity that creature had, until I booted her out of the way and to',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 2560]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

json

• Dataset: json
• Size: 59,805 training samples
• Columns: anchor, positive, negative, pair_s1, pair_s2, and label
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	pair_s1	pair_s2	label
  type	string	string	string	string	string	int
  details	min: 346 tokensmean: 454.86 tokensmax: 512 tokens	min: 339 tokensmean: 455.88 tokensmax: 512 tokens	min: 329 tokensmean: 456.76 tokensmax: 512 tokens	min: 41 tokensmean: 464.68 tokensmax: 512 tokens	min: 42 tokensmean: 464.46 tokensmax: 512 tokens	0: ~49.00%1: ~51.00%

• Samples:

  anchor	positive	negative	pair_s1	pair_s2	label
  ere's a hard limit of 1920x1080 in the current driver. https://github.com/LIS/lis-next/issues/318		On a side note… credit % usage is talked about as one of the key factors in a good score. I just paid off one of my cards that was ~25% usage. The difference in my credit score… +2 points. That's it?! What kinds of things make for big jumps once you're in the 750+ club? or does it not matter at this point?		They don't want to destroy the school because hurt. They want to destroy evidence because accountability. That's my $.02
  settings issue, or if it’s a RWD car on tarmac… it’s often just that twitchy. I sometimes change my controller settings just for tarmac to make things more manageable.		> I don’t generally play with all the assists off, and it happens whatever car I’m driving or whatever stage. What are your assist settings? Also, does it feel like you spin off immediately, or do you feel like it's more of an issue of over-correcting? As a controller player, I had to do a combination of all three things: adjusting sensitivity/linearity, adjusting the tunings for more understeer in order to be more forgiving of the snappiness of controller steering, and improving technique. There are definitely more and less forgiving cars, cars with better and worse stock tunings, etc.		You only do it when you don’t need more than 2/3 (red) or 1/3(yellow) throughput on one of the belts, assuming your default is blue belts. It’s a situational trick that works well on slow crafting recipes. I will often put the *pro...	f when you're negotiating a job offer is to participate on the hiring side.
  icity, minimalism. And you can just use two separate accounts today anyways.		Twitter bootstrap is awesome. I wish I could justify redoing my site using it but right now too much else going on. If you are developing Facebook apps, also check out f-bootstrap. It's based on Twitter bootstrap, but the styles and colors are more similar to Facebook's: https://github.com/ckrack/fbootstrapp		Uh..have you never heard of IntelliJ IDEA or Eclipse? Or RubyMine? Or Netbeans even...They pretty much do everything you're talking about.

• Loss: main.MNRLTripletAuxWithPairBCELoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• weight_decay: 0.01
• num_train_epochs: 1
• warmup_ratio: 0.05
• seed: 779
• bf16: True
• ddp_find_unused_parameters: True
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: {'use_reentrant': False}
• ddp_timeout: 7200
• batch_sampler: no_duplicates

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.05
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 779
• data_seed: None
• jit_mode_eval: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: True
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: {'use_reentrant': False}
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 7200
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 3.3.1
• Transformers: 4.57.0
• PyTorch: 2.11.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}