OBLIQ-IR-3B-no-distill
This is an ablation model, not the main system. It is OBLIQ-IR trained without the stylometric
kNN-graph distillation — the OBLIQ-IR (no distillation) row in every table of
OBLIQ-IR: Training a Dense Retriever for Oblique Queries (EMNLP 2026).
👉 For the actual system, use DataScience-UIBK/OBLIQ-IR-3B.
What exactly is removed
The name is short, so to be precise about it: this model is fully trained. It sees the complete per-mechanism synthetic mixture — 149,361 rows across all five tasks, with BM25 hard negatives, the same backbone, the same LoRA configuration, the same optimiser, the same schedule.
The only difference is that the 5,000 authorship kNN-graph pairs are absent from the writing subset. Nothing else changes. It is an ablation of one ingredient, not an untrained or partially-trained model.
Why it exists
It isolates what the kNN-graph distillation contributes. Removing those 5,000 rows costs Writing-Style 0.115 NDCG@10 while barely moving the other three tasks — which is the paper's central claim, and the reason the distillation is applied to writing only.
| NDCG@10 Gold | Writing | Math | Congress | |
|---|---|---|---|---|
| This model (no distillation) | .096 | .148 | .158 | .196 |
| OBLIQ-IR (dense) | .211 | .140 | .151 | .187 |
| Difference from distillation | +.115 | −.008 | −.007 | −.009 |
Distillation buys a large gain on the one task whose latent attribute has no topical footprint, and costs under 0.01 on the three where a topical lens already works.
Usage
Identical to the main model:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("DataScience-UIBK/OBLIQ-IR-3B-no-distill", trust_remote_code=True)
q = model.encode(["query: " + "your query"])
d = model.encode(["passage: " + "your document"])
scores = model.similarity(q, d)
The query: / passage: prefixes are required. Do not prepend a task instruction — the model was
neither trained nor evaluated with one.
A note on precision
These are merged full weights: W + BA·scaling folded into bf16. bf16 carries only about three significant
digits, so folding is slightly lossy and this repository does not reproduce its source adapter bit-for-bit.
Measured on this checkpoint:
| NDCG@10 Gold | This merged repo | Source adapter (the paper's numbers) |
|---|---|---|
| Math | 0.1485 | 0.1478 |
| Writing | 0.0952 | 0.0956 |
Small enough to be irrelevant for use, large enough to move the third decimal — Writing displays as .095 here against the paper's .096. The table above quotes the paper's adapter-form numbers.
Links
- 🤖 Main model —
OBLIQ-IR-3B - 🧩 Main adapter —
OBLIQ-IR-3B-LoRA - 📊 Data and runs —
OBLIQ-IR-Data - 💻 Code — github.com/DataScienceUIBK/obliq-ir
- 📄 Paper — OBLIQ-IR: Training a Dense Retriever for Oblique Queries
Licence
CC BY-NC 4.0. Redistributes weights derived from nvidia/llama-nv-embed-reasoning-3b, which NVIDIA
releases for non-commercial / research use (LICENSE_nvidia_base_model.txt, NOTICE.txt).
Built with Llama — the base derives from meta-llama/Llama-3.2-3B and the Llama 3.2 Community License
also applies.
Citation
@inproceedings{abdalla2026obliqir,
title = {{OBLIQ-IR}: Training a Dense Retriever for Oblique Queries},
author = {Abdalla, Mahmoud and Abdallah, Abdelrahman and Sedek, Shaimaa and Jatowt, Adam},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
year = {2026}
}