kenning-large-v0.4

Kenning is a System One decision model: given program state and typed questions (noul yes/no, choice, score) it returns typed answers with calibrated probabilities in one forward pass. It does not generate text. It was built with SystemOne Builder.

Model

• Base model: MoritzLaurer/deberta-v3-large-zeroshot-v2.0-c (MIT; fine-tuned on MNLI, FEVER-NLI (CC-BY-SA-3.0) and Mixtral-generated synthetic data (no non-commercial data))
• Architecture: cross-encoder; every candidate answer is scored against the state, and each question's distribution is softmax(scores / T) with per-type temperatures fitted on held-out data
• Temperatures: {"score": 1.1618, "choice": 1.2214, "noul": 1.2214}
• Max tokens per (state, answer) pair: 512
• Created: 2026-10-03T06:41:23Z

Training data

Distilled: every label was blended with the soft labels (probabilities) of Cloudflare/clef-flash (Apache-2.0); the target is 50% original label + 50% teacher.

Held-out results (split of the training pool)

These are in-distribution numbers. Benchmark on your own data (and recalibrate on a few hundred labelled examples from it with systemone calibrate) before letting the model act automatically.

General benchmark (headline)

systemone bench --suite general: 1,328 held-out items, 30 questions in 7 families of state. Labels come from public held-out datasets, or from exact rules for generated records, logs and agent steps; no model labels them. The score is macro accuracy (the mean over questions). Both models ran on one RTX 3090, 5/5 identical on repeat.

Kenning is close to Clef on text, about 20 times smaller and 4 times faster. On structured state (records, tables, agent steps, logs) it is far behind and often near chance. If your state looks like that, measure it on your own data first, or train it for your problem. Per-task results and method: Kenning docs.

Email suites

Recorded with systemone bench on suites never used for training. Automated is the share of items decided without a person (p >= 0.9 or <= 0.1); a threat auto-closed is a positive item the model was sure was negative.

Where a suite asks several questions, accuracy lists each (out of domain: spam / emotion / news topic; layouts: one per layout). Comparisons with Cloudflare Clef and TypeSafe Jev on the same suites are in the Kenning docs.

Use

pip install "systemone-client[local]"

from systemone import Kenning, Noul, Choice

model = Kenning.from_pretrained("systemonedev/kenning-large-v0.4")  # in-process, runs on CPU or GPU
answer = model.system_one(state={"ticket": "I was charged twice."},
                          questions={"billing": Noul("Is this about billing?")})
print(answer.nouls["billing"].noul)

Or serve it with SystemOne Builder's kenning service and call POST /v1/systemone with systemone.Client.

Limitations

• Structured state is the weak spot. Rules over records, numbers and dates, tables, agent steps and logs score 0.29–0.55 on the general benchmark (Clef: 0.74–0.86). Its training data was mostly short text.
• 512 tokens per (state, answer) pair. Longer states are truncated, so a decision that depends on the end of a long log or document is not seen.
• Calibration was fitted on the training distribution; probabilities on other data are scores until recalibrated (systemone calibrate refits the temperatures on your labelled data).
• Answer-quality judgements (is an answer helpful or correct) are hard for it and for Clef.
• Determinism: identical requests give identical answers on the same hardware and software; results can differ across GPUs or library versions.

Licence

The weights are released under the Apache License 2.0 (LICENSE). Upstream licences of the base model and of every training source are listed in NOTICE.md; some are share-alike (CC-BY-SA-3.0), so keep NOTICE.md with the weights.

Kenning implements a wire format compatible with TypeSafe AI's System One API. It is not affiliated with or endorsed by TypeSafe AI, and was not trained on TypeSafe outputs.