LRAT Qwen3-Embedding-0.6B Strong768
Full-parameter dense retriever trained from Qwen/Qwen3-Embedding-0.6B.
Training
| Stage | Data | Query length | Passage length | Epochs | Learning rate |
|---|---|---|---|---|---|
| 1 | LRAT clean-v1 | 512 | 512 | 2 | 1e-6 |
| 2 | same-row hard negatives | 512 | 768 | 1 | 2.5e-7 |
Common settings:
- one positive and nine negatives per query
- weighted multi-positive InfoNCE
- temperature
0.02 - last-token pooling and L2 normalization
- BF16, DDP, gradient checkpointing and GradCache
- global query batch size
256 - seed
2025
Query format:
Instruct: Given a web search query, retrieve relevant passages that answer the query
Query:{query}
Hard negatives are selected only from negative candidates in the same official LRAT row. No external datasets, additional trajectories, teacher models, API labels or generated documents are used.
Checkpoint
- architecture: Qwen3
- parameters:
595,776,512 - embedding dimension:
1024 - tokenizer vocabulary: unchanged
- full model: yes
model.safetensorsSHA-256:4a9cfd26263fea362943448e5ac2af37457ec11d062178fe490cde7b5958ecfb
Training code: https://github.com/yhcycdl/lrat-retriever