LRAT Qwen3-Embedding-0.6B — Marginal Utility Weighted
This retriever is trained from the released LRAT trajectory positives using a two-stage supervision pipeline:
- Retain the original LRAT reasoning-aware binary relevance filter.
- Score every retained positive from 0 through 4 for its marginal contribution to the agent's task state, using Qwen3.5-4B and the fixed utility rubric.
Utility scores map to raw weights {0: 0.50, 1: 0.75, 2: 1.00, 3: 1.50, 4: 2.00}. The raw weights are divided by their global dataset mean before
training. The objective is weighted InfoNCE,
sum_i(w_i * loss_i) / sum_i(w_i). No pairwise loss or reasoning-length weight
is used.
Training recipe
- Base model:
Qwen/Qwen3-Embedding-0.6B - Dataset: organizer-provided
Yuqi-Zhou/LRAT-Trainpositives and negatives - Epochs: 2
- Per-device batch size: 32
- Group size: 10 (one positive, nine trajectory negatives)
- Learning rate:
1e-6 - Warmup ratio:
0.1 - Query and passage maximum length: 512
- Temperature:
0.02 - Pooling: last token
- Normalized embeddings: yes
- Cross-device negatives: yes
- Precision: BF16
- Seed: 42
Query instruction:
Instruct: Given a web search query, retrieve relevant passages that answer the query
Query:{query}
The repository contains a full Transformers checkpoint and tokenizer files.