LRAT Qwen3-Embedding-0.6B — Retriever-Observable Utility V2
This retriever is trained from the released LRAT trajectory positives using supervision aligned with the information available to the retriever:
- Retain the original LRAT reasoning-aware binary positive filter.
- Score each unique
(web search query, document)pair from 0 through 4 for standalone retrieval utility using Qwen3.5-4B. - Give identical query-document pairs one shared score.
- Remove an explicit negative when that document is a retained positive for the identical search query.
The utility judge does not use the agent's prior knowledge, visited-document history, post-browse reasoning, next action, or distance to the final task answer. In particular, a document is not downweighted merely because an agent had already observed the information.
Utility scores map to raw weights {0: 0.50, 1: 0.75, 2: 1.00, 3: 1.50, 4: 2.00}. Raw weights are divided by their global dataset mean. Training uses
weighted InfoNCE, sum_i(w_i * loss_i) / sum_i(w_i), with no pairwise loss or
reasoning-length weight.
Training recipe
- Base model:
Qwen/Qwen3-Embedding-0.6B - Dataset: organizer-provided
Yuqi-Zhou/LRAT-Train - Epochs: 2
- Per-device batch size: 32 on 4 GPUs
- Group size: 10 (one positive and nine trajectory negatives)
- Learning rate:
1e-6 - Warmup ratio:
0.1 - Query and passage maximum length: 512
- Temperature:
0.02 - Pooling: last token
- Normalized embeddings: yes
- Cross-device negatives: yes
- Precision: BF16
- Seed: 42
Query instruction:
Instruct: Given a web search query, retrieve relevant passages that answer the query
Query:{query}
The repository contains a full Transformers checkpoint and tokenizer files.