🇳🇵 Supernova TeraLLM Reranker V1
Supernova TeraLLM Reranker V1 is an experimental Nepali-focused Transformer reranker developed by the Supernova project.
It scores the relevance of a document with respect to a query and is intended for search and retrieval pipelines.
Status: Production baseline / research model
🚀 Model Overview
The production architecture is a Transformer-based pair reranker.
Query + Document
↓
Supernova Tokenizer
↓
Token IDs
↓
Embedding
↓
16 Transformer Encoder Layers
↓
CLS Representation
↓
Ranking MLP
↓
Relevance Score
🧠 Architecture
Verified production architecture:
Embedding
├── Vocabulary size: 4
└── Hidden size: 1024
Transformer Encoder
├── Layers: 16
├── Hidden size: 1024
├── Feed-forward size: 4096
├── Multi-head self-attention
└── Dropout: 0.1
Ranking Head
├── Linear: 1024 → 512
├── GELU
├── Linear: 512 → 1
└── Sigmoid
Parameters
202,593,281 parameters
Production checkpoint size:
Approximately 91.96 MB
🔤 Tokenizer
The production tokenizer was verified against the model.
Model vocabulary: 4
Tokenizer vocabulary: 4
Embedding dimension: 1024
Compatibility test:
Input shape: (1, 128)
Maximum token ID: 3
Vocabulary size: 4
The generated token IDs are compatible with the model embedding.
📚 Confirmed External Dataset
Bharat-NanoMSMARCO Nepali
The confirmed external Nepali retrieval dataset used during the Supernova retrieval experiments is:
carlfeynman/Bharat_NanoMSMARCO_ne
Dataset:
https://huggingface.co/datasets/carlfeynman/Bharat_NanoMSMARCO_ne
Verified structure:
Corpus: 5,043 documents
Queries: 50 queries
Qrels: 50 relevance judgments
Language: Nepali
Task: Information Retrieval
License: CC-BY-4.0
The dataset contains Nepali retrieval queries, a document corpus, and query-document relevance judgments.
Users must follow the original dataset's license and attribution requirements.
🧪 Supernova Fine-Tuning Data
A later fine-tuning experiment constructed derived ranking triplets from the available retrieval material.
Random triplets: 1,000
Hard-negative triplets: 500
Total: 1,500
Split:
Training: 1,350
Validation: 150
These triplets are derived experimental examples, not a separate public dataset.
⚔️ Supernova vs Jina Reranker V3
A custom three-example live benchmark was performed.
Categories:
- Historical Nuance
- Geographical Logic
- Slang / Intent Extraction
Results
Category Supernova V1 Jina-v3
---------------------------------------------------
Historical Nuance PASS PASS
Geographical Logic PASS PASS
Slang / Intent FAIL FAIL
---------------------------------------------------
Pairwise Accuracy 2/3 2/3
66.67% 66.67%
Mean Positive-Negative Margin
Supernova V1: +0.001901
Jina Reranker V3: +0.140182
Experimental CPU Latency
Supernova total: 8.5254 seconds
Jina-v3 total: 88.3780 seconds
Supernova/example: 2.8418 seconds
Jina-v3/example: 29.4593 seconds
These measurements are specific to the experimental environment.
⚠️ Benchmark Interpretation
The three-example benchmark resulted in an accuracy draw:
Supernova V1: 2/3
Jina-v3: 2/3
Therefore this benchmark does NOT establish that Supernova V1 beats Jina Reranker V3.
Jina-v3 produced a substantially larger mean ranking margin in this particular test.
A substantially larger independent benchmark is required for strong performance claims.
🎯 Intended Use
Supernova V1 can be used for:
- Nepali search
- Semantic retrieval
- Search-result reranking
- Document relevance scoring
- RAG pipelines
- Nepali information retrieval research
- Multilingual retrieval research
Typical pipeline:
User Query
↓
First-Stage Retriever
↓
Top-K Documents
↓
Supernova Reranker
↓
Re-ranked Results
↓
Search / RAG
🛑 Limitations
- The confirmed Nepali retrieval dataset is relatively small.
- The Jina comparison contains only three examples.
- The benchmark is not a large standardized reranking benchmark.
- Results can vary by domain and query distribution.
- The current benchmark does not prove superiority over Jina-v3.
- Larger held-out evaluation is required.
- More hard-negative mining is required for stronger generalization.
🔬 Future Research
Future Supernova reranker versions will investigate:
- Larger Nepali retrieval datasets
- More diverse Nepali queries
- Stronger hard-negative mining
- Better query-document interaction
- Parameter-efficient fine-tuning
- Improved ranking objectives
- Larger held-out benchmarks
- CPU-efficient inference
- Multilingual retrieval
- Better generalization
🔁 Reproducibility
The published V1 checkpoint should remain the immutable baseline.
Future experiments should create a separate fine-tuning copy:
Supernova V1
↓
Immutable baseline
Supernova V1
↓
Fine-tuning copy
↓
Nepali ranking data
↓
Fine-tuning
↓
Held-out evaluation
📦 Production Checkpoint
File: pytorch_model.bin
Parameters: 202,593,281
Size: ~91.96 MB
📜 Dataset Attribution
Bharat-NanoMSMARCO Nepali:
https://huggingface.co/datasets/carlfeynman/Bharat_NanoMSMARCO_ne
Please consult the original dataset card for complete licensing and attribution requirements.
📖 Citation
@misc{supernova_teraillm_reranker_v1,
title={Supernova TeraLLM Reranker V1},
author={Supernova},
year={2026},
publisher={Hugging Face},
note={Nepali-focused Transformer reranker}
}
⚖️ Disclaimer
This model is provided for research and experimental purposes.
Benchmark results represent measurements from the development environment and are not universal guarantees of performance.
Users should evaluate the model on their own data and deployment environment before production use.
🇳🇵 Supernova
Building AI infrastructure for Nepali language technology.