LightDec_V2 (Long)

View in Model Surgeon

LightDec_V2 is a small, fast, calibrated decision model from Falcons.ai. Given a piece of state (an email, a ticket, an agent trace, a contract, a log, a JSON record) and one or more typed questions, it picks an answer from a closed set of options in a single forward pass and returns calibrated probabilities, so your application knows when to trust the answer and when to defer to a human or a larger model.

It is the long-context successor to Falconsai/LightDec. Every call now uses a 2,048-token window (LightDec v1 used 512 tokens by default and only switched to 2,048 for questions with more than 24 options), so long email threads, contracts and service logs fit without truncation.

What it does

LightDec_V2 answers the kind of small, structured questions that sit inside real systems: Which queue does this ticket go to? Does this email ask for a refund? How angry is the customer, 0–2? Is this agent step wrong? Is this prompt a jailbreak? Does this invoice violate the policy? It is not a generative model and never produces free text; it only chooses among the options you give it, which makes its output easy to validate, log and act on.

Three question types are supported, and several questions about the same state can be asked in one call:

• choice: pick one of N labelled options (options can carry descriptions, e.g. {"billing": "Charges, invoices, refunds"}).
• noul: a yes/no judgment about a statement (returns p_true).
• score: an ordinal scale such as ["Calm", "Annoyed", "Furious"] (also returns an expected_level).

Every answer includes a confidence and a defer flag. With the default defer_threshold of 0.7, the model answered 68.0% of test questions and was 90.6% accurate on those; the remainder are flagged for review.

How it works

The question, the options and the state are packed into one sequence:

[CLS] question [SEP] [MASK] option_1 [MASK] option_2 ... [MASK] option_k [SEP] state [SEP]

The encoder reads the whole sequence once. The hidden vector at each [MASK] marker represents one option; it is combined with the [CLS] context and a question-type embedding, then passed through a 2-layer, 8-head set transformer in which options attend to each other without positional encoding, so the result does not depend on option order. An MLP produces one logit per option and a softmax turns them into a distribution. Finally a temperature, learned per question type and per option-count bucket (2, 3–5, 6–12, 13+ options) and stored in falcondec_config.json, calibrates the probabilities.

Questions with more than 96 options are resolved with a tournament: options are scored in chunks, the best of each chunk advance, and a final pass ranks the survivors.

Usage

The model ships with its own loader, falcondec_modeling.py, which requires torch, transformers, safetensors and huggingface_hub.

import importlib.util
from huggingface_hub import hf_hub_download

path = hf_hub_download("Falconsai/LightDec_V2", "falcondec_modeling.py")
spec = importlib.util.spec_from_file_location("falcondec_modeling", path)
fd = importlib.util.module_from_spec(spec)
spec.loader.exec_module(fd)

model, tok = fd.load_falcondec("Falconsai/LightDec_V2")   # GPU if available, else CPU

state = {
    "email": {
        "subject": "Charged twice this month",
        "body": "I was billed twice for my Pro plan in March. Please refund the duplicate charge. This is the second time!",
    },
    "customer": {"plan": "Pro", "customer_since": "2021"},
}

questions = {
    "topic": {
        "type": "choice",
        "instructions": "What is this support email primarily about?",
        "criteria": {
            "billing": "Charges, invoices, payments, refunds or subscription costs",
            "technical": "Something in the product is broken, failing or erroring",
            "feature_request": "Asking for a new feature or an improvement",
            "account": "Login, password, profile, seats or account settings",
            "none": "None of these",
        },
    },
    "refund": {"type": "noul", "instructions": "The email asks for money to be returned or credited"},
    "anger": {"type": "score", "instructions": "How angry does the customer sound?",
              "criteria": ["Calm", "Annoyed", "Furious"]},
}

out = fd.decide(model, tok, state, questions)
for key, r in out["answers"].items():
    print(key, r["choice"], round(r["confidence"], 3), "DEFER" if r["defer"] else "")

Each result contains choice, choice_text, confidence, the full probs distribution and defer; noul answers add p_true and score answers add expected_level. Pass defer_threshold= to decide to trade coverage for accuracy. For lower-level control, fd.score_items(model, tok, items) scores raw {"state", "question", "options", "type"} items in batches.

CPU / small footprint. If the repository includes the int8 export (as LightDec v1 does in compact-int8/), load that folder the same way; the loader dequantizes the per-channel int8 weights automatically.

Evaluation

All numbers come from falcondec_report.json. The test split has 31,990 items across 73 tasks in 11 domains. Tasks marked held-out are flagged as such in the training report.

Overall

By domain

Head-to-head against the proof_v2 baseline

On 72 shared tasks (up to 120 items each), LightDec_V2 averaged 79.6% against 46.7% for the proof_v2 baseline, winning on 70 tasks, tying on 1 and losing on 1 (policy/invoice_total_transfer).

Support-email triage (bundled benchmark)

benchmarks/triage_support_email_test.jsonl contains 101 support emails, each with five typed questions and reference answers. It doubles as a worked example of the input format.

All five answers together route an email to the correct handling pile 94.1% of the time.

Long context

Three long-document tasks exercise the 2,048-token window (150 items each): long/contract_clause, long/email_thread and long/service_log. LightDec_V2 scored 100% on all three. These are synthetic, in-distribution tasks, so treat them as a check that long inputs are read end to end rather than as a measure of general long-document reasoning.

TEV1 transfer benchmark

TEV1 is a separate decision benchmark. Some of its tasks draw on the same public sources as LightDec's training data; the table marks which.

Overall TEV1 accuracy is 58.4% (2,800 items), and 51.8% on the tasks with no source overlap. For reference, the report lists published results for the 4B-parameter TEV1 model of 88% on its main decisions set and 100% on its policy transfer set; those are different splits and a model roughly 25× larger, so the figures are context rather than a like-for-like comparison.

Training

Validation accuracy peaked at epoch 4 while validation NLL kept rising after epoch 2, so the selected checkpoint is somewhat overconfident before calibration. The fitted temperatures (roughly 1.6 to 2.5) correct for this; they are part of the checkpoint and applied automatically by decide and score_items.

Speed

Measured with decide on a single state; latency grows only slightly as more questions are asked about the same state.

GPU throughput: about 2,589 decisions per second with batching.

Intended use

LightDec_V2 is built for high-volume, low-latency decisions inside software: ticket and email routing, intent detection, triage scoring, content and prompt-safety screening, policy and rule checks over structured records, agent step verification and "should the agent stop now" checks, and pre-filtering before a more expensive LLM call. The calibrated confidence and defer flag are designed for human-in-the-loop and cascade setups in which uncertain cases escalate.

Limitations

• Closed-set only. The model can only choose among the options you supply. If the right answer is not listed, it will still pick something; include a none option when that can happen.
• Weak at multi-step reasoning and arithmetic. Knowledge- and math-heavy tasks are well below the rest: MMLU 39.0%, AQuA-RAT 34.0%, ANLI 48.6%, ARC-Challenge 49.0%. Some policy tasks that require computing over a table are also weak: policy/table_count_transfer scores 32.4% with poor calibration (ECE 0.54), and policy/invoice_total_transfer (47.4%) is below chance. Use a larger model for anything that needs counting or totalling.
• Fine-grained and subjective labels. Accuracy drops on 77-way banking intents (58.0%), six-way emotion (47.2%) and five-level sentiment (SST-5 40.6%, Yelp 66.8%).
• Transfer to unfamiliar rule sets is limited. On TEV1 tasks without source overlap, accuracy is 51.8%, and routing and policy transfer (45.8% and 52.9%) are the weakest; validate on your own policies before relying on it.
• Guardrail coverage is partial. Jailbreak detection is strong (97.3%), but prompt-injection detection (59.5%) and harmful-request refusal (70.0%) are not reliable enough to be a sole safety layer.
• Calibration is aggregate. Overall ECE is low, but a few tasks (for example counsel/critique_quality, prompt_injections/detect, the table-count policy task) are noticeably miscalibrated; check calibration on your own distribution before tuning defer_threshold.
• English only, and questions are truncated to 96 tokens and each option to 24 tokens by default (option_tokens can raise the per-option budget).
• Long-context scores are synthetic. The 100% long-context results are in-distribution checks, not evidence of general long-document reasoning.

Files

Citation

@misc{falconsai_lightdec_v2_2026,
  title        = {LightDec_V2: A Long-Context, Calibrated Decision Model},
  author       = {Falcons.ai},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Falconsai/LightDec_V2}}
}

Built on the Ettin encoder from JHU CLSP (jhu-clsp/ettin-encoder-150m).

This card is generated from the surgical record itself; the package's lineage.intoto.jsonl is the signed source of truth (verify it free at the Surgeon's public verifier or with the bundled verify_attestation.py).

Architecture

• Identification: NLP · Small Language Model (SLM) (98% confidence)
• Source format: safetensors · Intended task: not declared
• config.json: synthesized from the anatomy (no source config.json)
• Source license: apache-2.0
• Lineage chain: 1 surgery (no prior attestation reachable) · Falconsai/LightDec_V2
• Post-surgery totals: 159,654,157 parameters · 168 tensors
• Compute estimate: 15.02439 GFLOPs (comparison metric, not a measurement)

Provenance & operations

• Parents: Falconsai/LightDec_V2/model.safetensors
• Operations performed: load×1
• Weight merges recorded: 0
• Quantized tensors (F32→F16): 0

Surgery Log (ordered)

1. load — hub:Falconsai/LightDec_V2/model.safetensors (319.3 MB, safetensors)

Validation

• Tissue imaging: not run
• Structural integrity is testable offline via the packaged load_and_test.py.

Compliance note

The signed attestation + this card together document model composition, modification history, and validation evidence — the record structure technical-documentation obligations (e.g. EU AI Act Annex IV) ask for. This is evidence, not legal advice.

Operated with Model Surgeon — verify this package at https://surgeon.falcons.ai/verify © 2026 FALCONS.AI — Model Surgeon record format. The model weights remain their owner's.