LightDec_Arthur

LightDec_Arthur is a small, fast, calibrated decision model from Falcons.ai. Given a piece of state (an email, a ticket, an agent trace, a log, a JSON record) and a typed question, it picks an answer from a closed set of options in a single forward pass and returns calibrated probabilities, so your application knows when to trust the answer and when to defer to a human or a larger model.

Unlike Falconsai/LightDec_V2, it has no pretrained backbone and no tokenizer. It reads raw UTF-8 bytes and was trained from scratch on the FalconDec_V2 decision mixture. At about 12 million parameters it is roughly 13× smaller than LightDec_V2, trading accuracy for size and speed.

What it does

LightDec_Arthur answers the same kind of small, structured questions as LightDec: Which queue does this ticket go to? Is this code a clone of that one? Is this prompt a jailbreak? Which intent is this request? It is not a generative model and never produces free text; it only chooses among the options you give it.

Three question types are supported:

• choice: pick one of N options.
• noul: a yes/no judgment (options Yes, No).
• score: an ordinal scale such as ["Calm", "Annoyed", "Furious"], trained with a ranked-probability term so near misses cost less than far ones.

The calibrated top probability works as a confidence. With a threshold of 0.80, the model answered 21.5% of test questions on its own and was 89.7% accurate on those; 94.6% of its mistakes fell in the deferred remainder.

How it works

The question, the options and the state are packed into one byte sequence:

[CLS] question [SEP] [OPT] option_1 [OPT] option_2 ... [OPT] option_k [SEP] state [SEP]

Each byte gets an embedding, plus hashed embeddings of the 2-, 3- and 4-byte fragments ending at it (65,536 buckets). A depthwise convolution mixes neighbouring bytes, and every 4 bytes are pooled into one position; each [OPT] marker starts a new position. One transformer block (8 heads, rotary positions) is applied 6 times over the text, with a learned step vector added before each pass.

The vector at each option's [OPT] position, plus the mean of the text and a question-type embedding, forms the option vector. The same block is applied twice more over the option vectors, without position encoding, so options can compare themselves with each other. A small MLP gives one logit per option. A temperature per question type and option-count bucket (2, 3–5, 6–12, 13+ options), fitted on validation data and stored in config.json, calibrates the softmax.

Usage

This repository does not yet include a standalone loader. The model definition (Arthur), the byte layout (assemble) and the temperature step live in the Arthur training notebook. To score a decision with the notebook's code:

import json, numpy as np, torch
from safetensors.torch import load_file

cfg = json.load(open("config.json"))                      # tier config, byte layout, temperatures
net = Arthur(TIER_CFG["base"])                             # class from the Arthur notebook
net.load_state_dict({k: v.float() for k, v in load_file("model.safetensors").items()})
net.to(DEVICE).eval()

d = dict(state="I was billed twice for my Pro plan in March. Please refund the duplicate charge.",
         question="What is this support email primarily about?",
         options=["billing", "technical", "feature request", "account"], qtype="choice")
ids, win, wmask, qt = batch_tensors([d])                   # from the Arthur notebook
with torch.no_grad():
    z = net(ids, win, wmask, qt)[0, :len(d["options"])].float().cpu().numpy()
t = np.array(cfg["temperatures"])[QTYPES[d["qtype"]], bucket(len(d["options"]))]
p = np.exp((z - z.max()) / t); p /= p.sum()
print(d["options"][int(p.argmax())], round(float(p.max()), 3), "DEFER" if p.max() < 0.8 else "")

Evaluation

All numbers come from results_full.json, written by the training run. The test split has 26,597 items across 58 tasks in 9 domains. Tasks marked held-out have no training or validation data.

By domain

Accuracy is the mean over the domain's tasks.

Head-to-head against Arthur nano

Arthur nano is the 1.6 MB tier of the same architecture (793,729 parameters), scored on the same test set. It was trained for 8 epochs with a 512-token window, so this comparison mixes model size, epochs and input length.

On the 58 shared tasks, LightDec_Arthur averaged 58.4% against 53.6% for Arthur nano, winning on 39 tasks, tying on 3 and losing on 16.

Input length

At the 1,024-token window, 385 of 972,625 training and test decisions (0.04%) have text cut off, concentrated in counsel/critique_quality (7.3%), jailbreak/detect (7.0%) and counsel/step_has_error (4.6%).

Decision Index 0.2 (partial)

LightDec_Arthur was scored on 39 benchmarks of the public Decision Index 0.2 with the index kit's own runner and scorers. Five benchmarks were not built (ToolRet and BRIGHT need tiktoken, POP909 needs mido, ChessBench needs python-chess, HLE is gated), so this is not a Decision Index score. The kit's rules apply: nothing is cut, and a request longer than 8,192 bytes, or a question of a type the harness does not map (score), is Unsupported and counts as wrong.

On 35 counted benchmarks, 21 score exactly 0 skill (chance level), and the best is SGD/SGD-X at 8.84. Five benchmarks share a source with the training mixture (ANLI, HellaSwag, WinoGrande, GSM8K, CLINC150), yet score near chance: the model learned those datasets in the training mixture's question format and does not transfer to the index's formats. For example, CLINC150 scores 84.8% on this card's test but 0.19 skill on the index, which asks all 151 intents plus out-of-scope in one question. 140 training decisions share text with index test rows (74 HellaSwag, 26 MMLU, 25 MMLU-Pro, 10 ARC-Easy, 4 ARC-Challenge, 1 BBH).

Training

Validation accuracy peaked at epoch 12 and drifted down slightly afterwards, while many late training batches reached near-zero loss.

Speed

Measured one decision at a time on 200 test decisions after 20 warm-up calls, including building the byte input.

GPU throughput: about 7,476 decisions per second in batches of 128. CPU latency has not been measured yet.

Intended use

LightDec_Arthur is built for high-volume, low-latency decisions where model size matters: ticket and email routing, intent detection, code and content screening, and pre-filtering before a more expensive model. At 24.5 MB it fits places a backbone-based model does not. Use the calibrated confidence to act on confident cases and send the rest to a person or a larger model.

Limitations

• Closed-set only. The model can only choose among the options you supply. If the right answer is not listed, it will still pick something.
• Weak outside its training formats. Held-out tasks average 44.2% against 65.4% for tasks seen in training, and on the Decision Index it scores near chance, even on benchmarks whose source it trained on. Validate on your own questions before relying on it.
• At or below chance on some tasks. Emotion 11.6% (chance 16.7%, and 38.6% for always giving the most common label), ANLI 32.1% (chance 33.3%), CommonsenseQA 19.1% (20.0%), HellaSwag 28.0% (25.0%), MMLU 28.4% (25.0%), ARC-Challenge 29.4% (25.0%), prompt-injection detection 47.4% and harmful-request refusal 50.7% (50.0%). It is not a safety layer.
• Fine-grained labels. 77-way banking intents score 4.7%.
• Policy transfer is uneven. Scores range from 100% (access_control) to below the most-common-answer baseline (invoice_overdue 48.8% against 61.6%, sla_urgency 30.2% against 40.2%). At confidence ≥ 0.80 it acts on 41% of policy decisions but is right on only 77.4% of those.
• Workflow decisions never reach 0.80 confidence (0 of 2,000 test items), so with that threshold they always defer.
• Calibration is aggregate. Overall ECE is 0.030; per-task calibration has not been measured.
• English only. Questions are cut to 96 tokens (384 bytes) and each option to 24 tokens (96 bytes) by default.
• No standalone loader yet; see Usage.

Files

Citation

@misc{falconsai_lightdec_arthur_2026,
  title        = {LightDec_Arthur: A Byte-Level, Calibrated Decision Model},
  author       = {Falcons.ai},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Falconsai/LightDec_Arthur}}
}

Architecture

• Byte-level input: 260 ids (256 bytes plus [PAD], [OPT], [SEP], [CLS]); no tokenizer
• Hashed 2/3/4-byte fragment embeddings: 65,536 buckets × 128 dimensions, projected to 512
• Depthwise convolution (kernel 5), then 4:1 pooling
• One transformer block (width 512, 8 heads, MLP 1,536, rotary positions) applied 6 times over the text and 2 times over the options
• Option scorer: 2-layer MLP, one logit per option
• 12,268,033 parameters, stored in fp16
• License: to be confirmed (training-data licence review pending)