local-decision-model
A small, fast decision model: unstructured state in, typed probabilistic decisions out. One forward pass of a 150M parameter encoder answers a whole schema of typed questions about a piece of text, typically in 10 to 25 ms on a consumer GPU. It never generates text, so every output is guaranteed to match its declared type.
Code, training recipe, experiments and full results: https://github.com/Pdbz199/local-decision-model
Usage
These weights need the small decisionmodel package from the GitHub repo (they are not a standard transformers head).
pip install git+https://github.com/Pdbz199/local-decision-model
from decisionmodel import DecisionModel, Bool, Enum, MultiLabel, Int, Extract
dm = DecisionModel.load("Pdbz199/local-decision-model")
d = dm.decide("My order #48213 arrived with a cracked screen. Refund me today or I dispute the charge.", {
"urgent": Bool("Does this need a response within the hour?"),
"team": Enum("Which team should handle this ticket?", ["billing and refunds", "technical support", "sales"]),
"anger": Int("How angry is the customer, from 1 (calm) to 5 (furious)?", 1, 5),
"order_id": Extract("What is the order number?"),
})
print(d.team.value, d.team.confidence, d.urgent.p(), d.order_id.value)
Types: Bool, Enum (2 to 255 options), MultiLabel, Int, Float, Extract (a verbatim substring of the input, or None).
How it works
ModernBERT-base with two small heads. The schema is written into the input as
[CLS] [MODE] question [OPT] option 1 [OPT] option 2 ... [SEP] state [SEP]. The hidden state at each [OPT] token is scored,
so all options are judged in the same pass; a span head handles extraction. Trained with log loss on about one million
examples recast from roughly 50 public datasets, then temperature-calibrated on tasks that were never trained on.
Results on tasks never seen in training
| Held-out task | Type | Result |
|---|---|---|
| IMDB sentiment | Bool | 94.4% accuracy, calibration error 0.016 |
| SQuAD v2 dev | Extract | 82.8% exact match, F1 85.8 |
| AG News topics | Enum (4) | 74.0% |
| Twitter financial news sentiment | Enum (3) | 74.1% |
| Banking77 intents | Enum (77) | 63.0% top 1, 80.2% top 3; 83.6% on the 48% of calls with confidence of at least 0.9 |
| deepset prompt injections | Bool | 69.6% accuracy, AUROC 0.87 |
| SMS spam | Bool | 64.9% with a vague question, 89.4% with a precise one (AUROC 0.94 to 0.96) |
Latency (RTX 5070 Ti, bfloat16, median): single yes/no 10 ms, 8 typed fields 16 ms, 77 options 11 ms, 255 options 21 ms, 8,000 token input 351 ms.
Limitations
- A small model: good at judgments a person could make at a glance, weak at implied or conditional meaning.
- Sensitive to question wording. Test a schema on a few real examples before trusting it.
MultiLabelunder-predicts when several options are true at once. Ask important tags as their ownBool.- Confidence is well calibrated on familiar kinds of task and can be overconfident on unfamiliar ones.
- English only. Guardrail decisions can be wrong: do not use it as a sole security boundary.
Licensing
The code is MIT licensed. The base model is Apache 2.0. These weights were trained on public datasets with mixed
licenses, some of which restrict commercial use (for example lmsys/toxic-chat, CC BY-NC 4.0, and the Yelp reviews
dataset). The weights are therefore released under cc-by-nc-4.0. If you need different terms, retrain with
scripts/build_data.py after removing the sources that do not fit your use; every source is listed in that file.
Relation to Jev
This project was inspired by TypeSafe AI's blog post introducing "System One Models" and their model Jev. It is an independent, generic implementation built only from that public post, and is not affiliated with TypeSafe AI.