TrailRAG Cross-Encoder: SciFact Enhanced

This is a fine-tuned cross-encoder model specifically optimized for Scientific Fact Verification tasks, trained as part of the PathfinderRAG research project.

Model Details

• Model Type: Cross-Encoder for Regression (continuous similarity scores)
• Base Model: cross-encoder/ms-marco-MiniLM-L-6-v2
• Training Dataset: SciFact (Scientific claim verification against research papers)
• Task: Scientific Fact Verification
• Library: sentence-transformers
• License: MIT

Performance Metrics

Final Regression Metrics

Training Details

• Training Duration: 32 minutes
• Epochs: 10
• Early Stopping: No
• Best Correlation Score: 0.689202
• Final MSE: 0.100697

Training Configuration

• Batch Size: 12
• Learning Rate: 1.5e-05
• Max Epochs: 10
• Weight Decay: 0.02
• Warmup Steps: 200

Usage

This model can be used with the sentence-transformers library for computing semantic similarity scores between query-document pairs.

Installation

pip install sentence-transformers

Basic Usage

from sentence_transformers import CrossEncoder

# Load the model
model = CrossEncoder('OloriBern/trailrag-cross-encoder-scifact-enhanced')

# Example usage
pairs = [
    ['What is artificial intelligence?', 'AI is a field of computer science focused on creating intelligent machines.'],
    ['What is artificial intelligence?', 'Paris is the capital of France.']
]

# Get similarity scores (continuous values, not binary)
scores = model.predict(pairs)
print(scores)  # Higher scores indicate better semantic match

Advanced Usage in PathfinderRAG

from sentence_transformers import CrossEncoder

# Initialize for PathfinderRAG exploration
cross_encoder = CrossEncoder('OloriBern/trailrag-cross-encoder-scifact-enhanced')

def score_query_document_pair(query: str, document: str) -> float:
    """Score a query-document pair for relevance."""
    score = cross_encoder.predict([[query, document]])[0]
    return float(score)

# Use in document exploration
query = "Your research query"
documents = ["Document 1 text", "Document 2 text", ...]

# Score all pairs
scores = cross_encoder.predict([[query, doc] for doc in documents])
ranked_docs = sorted(zip(documents, scores), key=lambda x: x[1], reverse=True)

Training Process

This model was trained using regression metrics (not classification) to predict continuous similarity scores in the range [0, 1]. The training process focused on:

1. Data Quality: Used authentic SciFact examples with careful contamination filtering
2. Regression Approach: Avoided binary classification, maintaining continuous label distribution
3. Correlation Optimization: Maximized Spearman correlation for effective ranking
4. Scientific Rigor: All metrics derived from real training runs without simulation

Why Regression Over Classification?

Cross-encoders for information retrieval should predict continuous similarity scores, not binary classifications. This approach:

• Preserves fine-grained similarity distinctions
• Enables better ranking and document selection
• Provides more informative scores for downstream applications
• Aligns with the mathematical foundation of information retrieval

Dataset

SciFact: Scientific claim verification against research papers

• Task Type: Scientific Fact Verification
• Training Examples: 1,000 high-quality pairs
• Validation Split: 20% (200 examples)
• Quality Threshold: ≥0.70 (authentic TrailRAG metrics)
• Contamination: Zero overlap between splits

Limitations

• Optimized specifically for scientific fact verification tasks
• Performance may vary on out-of-domain data
• Requires sentence-transformers library for inference
• CPU-based training (GPU optimization available for future versions)

Citation

@misc{trailrag-cross-encoder-scifact,
  title = {TrailRAG Cross-Encoder: SciFact Enhanced},
  author = {PathfinderRAG Team},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/OloriBern/trailrag-cross-encoder-scifact-enhanced}
}

Model Card Contact

For questions about this model, please open an issue in the PathfinderRAG repository or contact the development team.

This model card was automatically generated using the TrailRAG model card generator with authentic training metrics.