verdict-small
Try it in the browser: https://huggingface.co/spaces/Manav2op/verdict · Colab: https://colab.research.google.com/github/Manavarya09/verdict/blob/main/examples/verdict_quickstart.ipynb
The default encoder of Verdict: small, fast, honest
decision models. multilingual-e5-small (118M) fine-tuned on a typed-decision mix of 14
public datasets (intent, NLI, ordinal reviews, safety) so that cosine(input, option) × 20
is a good logit over a question's options. Banking77, SST-5 and ToxicChat were never in
the mix; they are the held-out zero-shot numbers below.
pip install verdictml
from verdict import Verdict
v = Verdict() # loads this model
v.choose("Billed twice, refund or we cancel", ["billing", "technical", "sales"])
v.check("Can I talk to a person?", claim="the user asks for a human")
Also runs in the browser via transformers.js (onnx/model_quantized.onnx, int8, 118 MB):
const extractor = await pipeline("feature-extraction", "Manav2op/verdict-small", { dtype: "q8" });
Held-out zero-shot (full test sets)
| suite | base e5-small | verdict-small |
|---|---|---|
| Banking77 (3,076) accuracy | 0.594 | 0.556 |
| SST-5 (2,210) accuracy | 0.274 | 0.403 |
| ToxicChat (5,083) AUROC | 0.59 | 0.892 |
With 16 labels per class and the package's heads: Banking77 0.842, ToxicChat AUROC 0.939.
Every number reproduces with python -m bench.run in the repo; protocol and all rows in
docs/BENCHMARKS.md.
Training: python -m train.train (repo), 2,000 steps, batch 32, lr 2e-5, Apple M5.
Data mix and caps: train/data.py. Prefixes: query: for inputs, passage: for options.