Sovereign Semantic Router v6 (JEPA lens)
A 41.4M-parameter latent trajectory predictor for multi-hop retrieval. Instead of generating text, it predicts the embedding of the next reasoning step directly in Harrier-270M space (dim 640), so retrieval becomes: predict → k-NN lookup → predict again.
It consumes vectors and returns a vector, so it drops into any existing retrieval stack. Inference runs on CPU in well under a millisecond.
Results
MuSiQue multi-hop retrieval — apples-to-apples against classical hybrid RAG on the same embedder, 2417 dev questions:
| Metric | Classical hybrid RAG | + this router |
|---|---|---|
| Complete Recall@10 | 20.19% | 28.59% (+8.4 pp) |
| MRR | — | 0.7671 |
| Recall@10 | — | 61.43% |
| At-Least-One@10 | — | 95.53% |
Complete Recall@10 means every supporting paragraph for a question landed in the top 10 — the metric that actually matters for multi-hop, and the one where single-shot retrieval fails.
No contamination. Training used MuSiQue split=train, evaluation uses dev. Verbatim text comparison: 0 overlapping questions, 0 of 2629 gold (supporting) dev paragraphs were ever encoded during training. Only distractor paragraphs (1.9%) and Wikipedia titles (45%) overlap.
Single-step latent prediction — on a corpus the model never saw during training, retrieving over 13,053 candidates:
| Predictor | hit@16 | hit@64 | recall@64 |
|---|---|---|---|
| Ridge regression trained on this corpus (1.2M params) | 0.396 | 0.536 | 0.907 |
| This router (never saw this corpus) | 0.430 | 0.527 | 0.937 |
| Previous turn, used as-is | 0.119 | 0.177 | 0.486 |
| Static hot cache (LRU analogue) | 0.003 | 0.008 | 0.005 |
Read this honestly: single-step prediction is a nearly linear problem, and a ridge regression fitted directly on the target corpus matches this model on hit@64. The router's edge is transfer (it reaches that level with zero exposure to the corpus) and ranking precision (hit@16, recall@64). Whether the non-linearity pays off in iterative multi-hop — where each step's input depends on the previous step's retrieval and errors compound — is the open question for v7.
Quick start
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("shalyhinpavel/sovereign-jepa-lens", trust_remote_code=True).eval()
# chain: (batch, steps, 640) — L2-normalized Harrier-270M embeddings
# step 0 = the query (encoded WITH the instruction prefix), steps 1+ = retrieved passages
chain = torch.nn.functional.normalize(torch.randn(1, 3, 640), dim=-1)
next_step = model.predict_next(chain) # (1, 640), unit-norm — feed this to your k-NN index
all_steps = model(inputs_embeds=chain).last_hidden_state # (1, 3, 640), prediction at every position
predict_next returns the predicted embedding of the following reasoning step: search your vector store with it, take what comes back, append it to the chain, repeat. That loop is the multi-hop retrieval this model exists for.
Verified on load: the wrapper reproduces the reference implementation bit-for-bit (max absolute deviation 0.0) and the reported cosine of 0.5320 on held-out data.
Input format
Position 0 is special. It holds a query encoded with the instruction prefix (Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: ...); positions 1+ hold raw texts encoded without it. This asymmetry mirrors production: stored documents are raw, incoming queries are instructed. Feeding uniformly-encoded vectors will degrade results.
Embeddings come from shalyhinpavel/harrier-v1-270m-gguf — this model consumes vectors, never text.
model.safetensors and semantic_router_v6.safetensors are identical weights under two names: the first is what from_pretrained looks for, the second is the filename requested by the Rust retrieval engine this router was built for.
Architecture
Transformer: 6 blocks, 10 heads, dim 640, RMSNorm, SwiGLU MLP (×4 expansion), learned positional embeddings (16 slots), 2-layer prediction head with a residual connection, L2-normalized output. Trained with InfoNCE (in-batch negatives) + Muon optimizer + an auxiliary two-hop MTP head that was dropped at checkpoint time.
The state dict contains an unused mask_token key — a training artifact, ignored at inference.
Limitations and provenance
Stated plainly, because these affect what you can reproduce:
- The weights are canonical; the training run is not reproducible. The training script and the exact training corpus were lost (they lived on an ephemeral cloud instance). These weights are the only surviving copy. A reproducible v7 — corpus built by a versioned script, trainer in git, split by source chain — is in progress.
- Do not cite earlier reported figures (Avg Sim 0.8393, Recall@10 95.7%, FRAMES 97%). Their measurement corpus did not survive and they do not reproduce on any available corpus; the numbers in the tables above are what has actually been re-measured, in August 2026, on this machine.
- Embedder-specific. This model operates strictly in Harrier-270M GGUF space. Vectors produced by sentence-transformers are not compatible — measured degradation is severe (hit@64 drops from 0.527 to 0.114).
- Quantization matters for reproducing the benchmarks. MuSiQue numbers were produced with the
Q8_0embedder build; the engine CLI defaults toQ4_K_M. Use Q8_0 to match.
Citation
@software{sovereign_jepa_lens_2026,
author = {Shalyhin, Pavel},
title = {Sovereign Semantic Router v6: latent trajectory prediction for multi-hop retrieval},
year = {2026},
url = {https://huggingface.co/shalyhinpavel/sovereign-jepa-lens}
}