canho/qwen3-emb-0.6b-reasonir-n2506-ckpt40
Qwen/Qwen3-Embedding-0.6B full fine-tuned with ms-swift InfoNCE, checkpoint at step 40 of 375
(epoch 0.54; best eval-loss checkpoint of the run so far: checkpoint-40).
- Training data: reasonir/reasonir-data hq + vl pooled, 2,506 rows sampled with seed 42 (695 hq + 1,811 vl; 1 positive + 1 synthetic hard negative; hq positives filled from xlangai/BRIGHT documents, so hq overlaps the BRIGHT corpus). Data license cc-by-nc-4.0
- Recipe: Qwen3-Embedding SWIFT.md (
swift sft --task_type embedding --loss_type infonce), lr 6e-6 (cosine), 5 epochs, 2 GPUs x per-device 4 x grad-acc 4 = 32 queries/step, in-batch negatives across the 8 queries of a micro-batch, temperature 0.1, DeepSpeed ZeRO-3, bf16, 5% held out for eval, checkpoint every 20 steps, seed 42. - Query prompt (set in
config_sentence_transformers.json, prepended to every query): 'Query:'; documents without a prompt.
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("canho/qwen3-emb-0.6b-reasonir-n2506-ckpt40", processor_kwargs={"padding_side": "left"})
q = m.encode(["..."], prompt_name="query"); d = m.encode(["..."])