engram-xenc-v0
Cross-encoder for Engram's substitute-hit gating layer. Given a user query and a candidate knowledge object (KO) body, predicts whether the KO answers the query well enough to substitute for a full LLM call.
Trained by distilling Claude Sonnet's ACCEPT/REJECT judgments on 1,560 (query, KO) candidate pairs sampled from the Bitext customer-support dataset.
Engram context
- Codebase consumer:
sidecar/gating_cross_encoder.py— invoked on retrieval candidates that pass an initial cosine-distance cut. A positive score routes the query to substitute; a negative routes to grounded-LLM. - Fetch via:
scripts/fetch_models.shin the Engram repo (idempotent post-clone install step). - Reproduce locally:
python tools/session8_5_train_eval.py(~15 minutes on M4 Max). Inputs aredata/session8_5_labeled.json(1,560 rows; 1,248 train / 312 test deterministic split). Deterministic seed (RANDOM_SEED=42).
Eval metrics (on held-out 20% test split)
| Metric | This model | Baseline (1 − cosine) |
|---|---|---|
| ROC-AUC | 0.961 | 0.800 |
| Average precision | 0.931 | 0.617 |
| Best F1 (threshold = 0.8) | 0.889 | — |
| Inference latency | 6.5 ms/pair | — |
AUC improvement of +0.161 over the cosine baseline; the model earns its keep specifically in the ambiguous-distance zone where cosine alone over-substitutes.
Numbers from data/session8_5_eval.json at training time
(2026-04-30). Stochastic MPS ops mean re-trains land within
~±0.01 AUC, not bit-identical.
Training details
- Base model:
cross-encoder/ms-marco-MiniLM-L-12-v2(12-layer cross-encoder pre-trained on MS MARCO passage ranking). - Loss: binary cross-entropy on the verifier labels (ACCEPT=1, REJECT=0).
- Hyperparameters: 3 epochs, batch size 16, 100 warmup steps.
- Wall time: 869.6 s on Apple M4 Max via MPS backend.
Intended use
- Best for: customer-support / FAQ retrieval where a finite knowledge base covers most queries and the system needs to decide whether to answer from the KB at all.
- Out of scope: open-domain QA, long-form generation, anything needing more than the top-1 KO. This is a gating signal, not a generator.
Limitations
- Domain: trained on customer-support queries (Bitext). Will
transfer poorly to legal, medical, or technical-research
domains without re-training. The Engram repo's
tools/session8_5_train_eval.pyis designed to be re-run per customer corpus. - Calibration: the 0.8 F1-optimal threshold is corpus-specific. Re-calibrate against your own labels before relying on it.
- No PII / safety filter. Inherits the base model's behavior.
License
Apache 2.0. Derived from cross-encoder/ms-marco-MiniLM-L-12-v2
(Apache 2.0). Training data: Bitext customer-support dataset
(verify upstream license at
huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset
before any commercial use).
Citation
@misc{engram-xenc-v0,
author = {Pascaline / Engram},
title = {engram-xenc-v0: cross-encoder for KO substitute-hit gating},
year = {2026},
url = {https://huggingface.co/jchiang11/engram-xenc-v0}
}