decide-0.2.0

A small, fast, non-autoregressive decision model. Given some state, an instruction and a set of candidate options, it scores every option in a single forward pass and returns calibrated probabilities. It answers three kinds of question with the same weights:

• choice: a distribution over K mutually exclusive options
• noul: the probability that a condition holds (a yes/no question)
• score: an expected level on an ordered scale

It is fine-tuned from ModernBERT-base (149M parameters). The packed input is <instructions> <state> [SEP] [MASK] option0 [MASK] option1 ...; a small MLP head scores the encoder state at each [MASK]. Each option attends only to the shared prefix and to itself and its position ids restart after the prefix, so an option's score depends on (premise, that option) alone: permuting the options permutes the scores and changes nothing else (measured max logit change under permutation: 8.9e-06).

The model, tokenizer, training loop, evaluation and this card were all produced with the pure-Go toolchain in jmwri/decide; no Python or PyTorch was used to train it.

Usage

import "github.com/jmwri/decide"

decide.SetDefault(decide.NewLocal("/path/to/this/model")) // or: decide pull --repo jmwri/decide
ans, _ := decide.Decide(ctx, "Replication lag exceeded 45 seconds.", decide.Options{
    {Key: "infrastructure", Description: "Database, hardware, network or server failures"},
    {Key: "billing", Description: "Invoices, payments, refunds"},
}, "Classify the root cause domain.")

Evaluation

Accuracy is top-option accuracy. ECE is the expected calibration error (15 bins) of the top-option confidence, after the per-kind temperature scaling shipped in decide.json.

Held-out validation splits of the training tasks

Tasks in this second table were never trained on (zero-shot to the model):

Held-out tasks

Training data

Fine-tuned on 301661 examples built from the public datasets below (options are re-sampled and verbalised in several styles; classification tasks are also posed as yes/no verification questions). Licenses are as declared on the Hugging Face hub; review them before redistributing or commercialising the weights, since several datasets carry share-alike or unspecified terms.

Limitations

• English only. Inputs are limited to 8192 tokens; option sets are best kept under ~10 (use a two-stage taxonomy beyond that).
• Trained on public classification, inference and multiple-choice datasets, so it is strongest on those styles of question. Explicit, descriptive option texts work better than bare labels. Domain-specific jargon or rubrics with no descriptive criteria fall back to general language priors.
• Probabilities are calibrated on held-out data from the training tasks; on very different distributions they will be over-confident. Use a confidence gate and route uncertain cases to review.
• Not for decisions that affect people's rights, safety or livelihood without human oversight.

License

Weights: Apache-2.0 (ModernBERT-base is Apache-2.0), subject to the training-data terms above.