canho/qwen3-emb-0.6b-ar-ours6k-ckpt310
Qwen/Qwen3-Embedding-0.6B full fine-tuned with ms-swift InfoNCE, checkpoint at step 310 of 815
(epoch 1.91; best eval-loss checkpoint of the run so far: checkpoint-310).
- Training data: HF canho/ours-6k train split (5,469 pairs; query + 1 positive + 9 hard negatives)
- Recipe: Qwen3-Embedding SWIFT.md (
swift sft --task_type embedding --loss_type infonce), same recipe as the step-1 / ablation runs: lr 6e-6 (cosine), 5 epochs, 32 queries/step (8 GPUs x per-device 1 x grad-acc 4; earlier runs 2 x 4 x 4), in-batch negatives across the 8 queries of a micro-batch (embeddings all-gathered across GPUs), temperature 0.1, DeepSpeed ZeRO-3, bf16, 5% held out for eval, eval + checkpoint every 10 steps, seed 42. - Query prompt (set in
config_sentence_transformers.json, prepended to every query): 'Query:'; documents without a prompt.
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("canho/qwen3-emb-0.6b-ar-ours6k-ckpt310", processor_kwargs={"padding_side": "left"})
q = m.encode(["..."], prompt_name="query"); d = m.encode(["..."])