LemonDecider

A typed decision engine derived from Qwen/Qwen3.5-4B: it answers structured choice / noul / score questions in a single forward pass with an explicit unknown (abstention) candidate. There is no autoregressive generation head — the backbone is kept full-weight, and a 255-way single-token readout replaces the LM head.

Model description

Each input is rendered as one user message (state + instructions + options + an appended __unknown__ slot), then scored in one forward pass. The forward pass rejects KV-cache decoding and right padding (left padding is required).

Question types and output semantics

decision_utils.py implements the native _known() semantics: probabilities over known candidates are renormalized, and the unknown slot gets its own probability.

confidence = concentration · (1 − P(unknown)), where concentration = (n·p_max − 1)/(n − 1) (1 for single-candidate questions).

Quickstart

Standalone CLI (bundled model.py):

python model.py --checkpoint . \
  --input inputs.json \
  --device cuda:0

With transformers:

import torch
from transformers import AutoModel, AutoProcessor
from decision_utils import predict_batch

model = AutoModel.from_pretrained(
    "inclusionAI/LemonDecider",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda:0",
    attn_implementation="sdpa",
).eval()
processor = AutoProcessor.from_pretrained(
    "inclusionAI/LemonDecider", trust_remote_code=True
)
processor.tokenizer.padding_side = "left"

rows = [{
    "state": "我想申请退款。",
    "question": {
        "type": "choice",
        "instructions": "判断主要诉求",
        "criteria": {
            "refund": "退款相关",
            "delivery": "物流相关",
            "account": "账户相关",
        },
    },
}]

result = predict_batch(model, processor, rows)
print(result["answers"][0])
# {"type": "choice", "choice": "refund", "probabilities": {...},
#  "confidence": ..., "unknown_probability": ..., "abstained": ...}

Images are passed as decoded PIL.Image objects in row["images"]; paths/URLs are not fetched.

Training

Distilled via supervised fine-tuning from imajev-4b, a larger Jev-class decision model:

• Checkpoint: step 24 (6,144 examples seen)
• Loss: cross-entropy over the masked 255-way readout (supports hard option indices and soft teacher distributions)
• Dev set (n=574) accuracy@T=1: 0.7735 (best step: 3, 0.7805)

Temperature fit (not applied): an NLL scan over T ∈ [0.30, 5.00] on the dev set found T = 0.60 (NLL 0.56265, Brier 0.28261, vs 0.6016 / 0.29885 at T=1). The checkpoint still serves at T = 1.0; adopting T* is deliberately blocked until the public evaluation suite is re-evaluated (argmax is invariant at T* except at the noul decision boundary).

Intended use and limitations

• Intended for typed decision inputs (choice / noul / score) rendered through decision_utils.decision_messages(); prompt-layout divergence will degrade results.
• It is not a generative chat model: never use it for free-form text generation.
• Results come from a small internal dev split (n=574); validate on your own distribution before production use.
• Long inputs are rejected above 8,192 tokens instead of being silently truncated; handle your own context budget upstream.

Files

License

Apache-2.0 (see LICENSE / LICENSE.code).