MindLens RAG Reranker (Cross-Encoder)
Model Description
Cross-encoder fine-tuned from cross-encoder/ms-marco-MiniLM-L-6-v2 (pre-trained on MS MARCO passage
ranking) on mental-health domain query-passage pairs to re-rank ChromaDB
retrieval results before passage injection into the Groq therapy-response
generator. This is Model 5 in the MindLens five-model NLP pipeline.
How It Works in MindLens
- User message arrives at the orchestrator
- ChromaDB retrieves top-20 candidate passages by cosine similarity
- This reranker scores each (user message, passage) pair jointly
- Top-3 ranked passages are injected into Groq Llama 3.3 70B's system prompt
- Groq generates the therapy response, grounded in the most relevant counselling text
Training Data
Built from nbertagnolli/counsel-chat
using a three-tier pair construction strategy:
- Tier 1 — Automatic positives: (question, its own counsellor answer) → label 1
- Tier 2 — Hard negatives: (question, answer from a different topic) → label 0
- Tier 3 — Groq-scored: (question, same-topic different-answer) scored by Groq Llama-3.1-8B as LLM-as-judge (Zheng et al., 2023) → binary label
Training pairs: 3,832 | Val: 479 | Test: 480 Groq-scored ambiguous pairs: 0
Evaluation Results (held-out test set)
| Metric | Score | Interpretation |
|---|---|---|
| NDCG@3 | 1.0000 | Ranking quality of top-3 results |
| MRR | 1.0000 | How quickly the first relevant passage is found |
| Precision@3 | 0.5833 | Fraction of top-3 results that are relevant |
Usage
from sentence_transformers import CrossEncoder
reranker = CrossEncoder("AmiruMallawarachchi/mindlens-rag-reranker")
query = "I feel completely hopeless and can't stop crying"
passages = [...] # top-20 from ChromaDB
scores = reranker.predict([[query, p] for p in passages])
top3_indices = scores.argsort()[::-1][:3]
top3_passages = [passages[i] for i in top3_indices]
Citation
Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023. Nogueira, R. & Cho, K. (2019). Passage Re-ranking with BERT.