rev-decision-model

Non-autoregressive, calibrated System 1 Decision Model. Evaluates typed questions (choice, score, noul) over any structured JSON, email, or customer state in a single forward pass (~33 ms) with zero text generation, zero decoding latency, and zero hallucination risk.

Trained with Reinforcement Learning from Calibrated Distributions (RLCD) against strictly proper scoring rules (LogScore + Spherical + Ranked Probability Score) on top of ModernBERT-large (421M parameters, 8192 context).

⚡ Key Capabilities

• Single Forward Pass (<35 ms): No autoregressive token-by-token decoding. All questions and options are evaluated simultaneously in parallel.
• Option Marker Pooling: Formats inputs with candidate option tokens ([MASK]), gathering option hidden states directly from the bidirectional transformer.
• Calibrated Probabilities (ECE ~0.08): Optimized under strictly proper scoring rules and post-hoc L-BFGS temperature scaling ($T \in [0.5, 5.0]$) to prevent pathological overconfidence.
• Built-in Action / Deferral Head (act_head): Automatically classifies whether an agent should act autonomously or defer/escalate to a human or System 2 LLM when predictions are close or ambiguous.
• Continuous Expected Score: For ordinal score questions, outputs continuous floating-point expected levels ($\mathbb{E}[ ext{score}] = \sum i \cdot p_i$) rather than lossy discrete argmaxes.
• TypeSafe API Compatible: Matches the POST /v1/systemone specification.

📐 Architecture

Input: State (JSON, Email, Ticket) + Questions (Choice, Score, Noul)
                               │
                               ▼
Sequence: [CLS] <type> instructions [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state [SEP]
                               │
                               ▼
            ModernBERT-large Backbone (421M, 8192 context)
                               │
            Gather hidden states at [MASK] marker positions
                               │
            2-Layer TransformerEncoder Head + Type Embedding (type_emb)
                               │
            ┌──────────────────┴──────────────────┐
            ▼                                     ▼
      Option Scorer Head                    Action / Deferral Head
Linear(d,d) -> GELU -> Linear(d,1)        Linear(d+4, 256) -> GELU -> Linear(256, 2)
            │                                     │
      Calibrated Logits                     [Act Autonomously vs Defer]

Option Marker Pooling

Instead of running separate forward passes for each option or decoding tokens one-by-one, the prompt injects a [MASK] marker before each option:

[CLS] choice question: Which team handles this? [SEP] [MASK] billing [MASK] infrastructure [MASK] sales [SEP] {"message": "Database is down"} [SEP]

The model gathers representations $h_m$ directly at each [MASK] token, allowing full bidirectional cross-attention between the state, instructions, and all candidate options.

🔬 Training: RLCD with Strictly Proper Scoring Rules

Standard cross-entropy training encourages overconfident predictions on ambiguous boundary samples. This model is trained with RLCD (Reinforcement Learning from Calibrated Distributions) using strictly proper scoring rules where expected reward is mathematically maximized if and only if the predicted probability distribution matches the true posterior:

$$\mathcal{R}(p, y) = S_{ ext{log}}(p, y) + w_{ ext{sph}} \cdot S_{ ext{spherical}}(p, y) - w_{ ext{rps}} \cdot ext{RPS}(p, y) \cdot \mathbb{I}_{ ext{score}}$$

1. Logarithmic Score: $S_{ ext{log}}(p, y) = \sum_{k=1}^K y_k \log p_k$
2. Spherical Score: $S_{ ext{sph}}(p, y) = rac{\sum_k y_k p_k}{|p|_2}$ (rewards peaked distribution only when aligned with target)
3. Ranked Probability Score (RPS): $$ ext{RPS}(p, y) = rac{1}{K-1} \sum_{k=1}^{K-1} (P_k - Y_k)^2$$ Where $P$ and $Y$ are cumulative CDFs. RPS heavily penalizes distant ordinal mistakes (e.g. predicting Severity 3 instead of Severity 0 is penalized far more heavily than predicting Severity 1).

Post-Training Calibration Temperatures

Fitted via L-BFGS on validation splits:

• choice: $T = 1.0346$
• score: $T = 0.8955$
• noul: $T = 0.9248$

📊 Benchmark Evaluation

Evaluated against the official LocalLLaMA/typed-decisions benchmark (400 cases, 2,000 decisions across Customer Service, Invoice Processing, Security Incident Response, and Agent Trace Observability):

🚀 Quickstart

1. Installation

pip install rev
# or install laya:
pip install laya

2. Python Inference

import rev

# Load model directly from Hugging Face
agent = rev.Agent("jaswanthsanjay88/rev-decision-model")

# 1. Provide any structured state (dict, JSON, or text)
state = {
    "ticket_id": "TCK-9812",
    "customer": "enterprise_corp",
    "message": "URGENT: Our production cluster is down and returning 502 errors across all nodes!"
}

# 2. Define typed questions (choice, score, noul)
questions = {
    "routing_team": {
        "type": "choice",
        "instructions": "Which engineering team should handle this incident?",
        "criteria": {
            "billing": "invoice and payment queries",
            "infrastructure": "site outages, kubernetes, cluster crashes",
            "sales": "upgrades and licenses"
        }
    },
    "priority": {
        "type": "score",
        "instructions": "Determine escalation priority level:",
        "criteria": ["low priority", "medium priority", "high priority", "critical p0 outage"]
    },
    "sla_breach_risk": {
        "type": "noul",
        "instructions": "Is this customer at immediate risk of SLA breach?"
    }
}

# 3. Single forward pass (<35ms)
result = agent.predict(state, questions)

print("Routing Team :", result["answers"]["routing_team"]["choice"])
# -> "infrastructure" (Confidence: 0.507, Margin: +0.227)

print("Priority     :", result["answers"]["priority"])
# -> "level 1: medium priority" (Continuous Expected Score: 1.623)

3. Automated Deferral to System 2 / Human

# The model's action head automatically flags when to act or defer:
for qid, ans in result["answers"].items():
    margin = ans.get("margin", 0.0)
    if margin < 0.15:
        print(f"⚠️ {qid} is ambiguous (margin={margin:.4f}). DEFERRING to on-call human / System 2 LLM.")
    else:
        print(f"✅ {qid} has high confidence margin. ACTING AUTONOMOUSLY.")

🛠️ Reproduction & Training

Full training and post-training temperature calibration can be reproduced using our open Google Colab / Kaggle notebook:

• Notebook: train_laya_system_one_decision_model.ipynb
• Runs on 2× NVIDIA T4 GPUs in ~5 minutes using torchrun --standalone --nproc_per_node=2.

⚠️ Limitations & Failure Modes

An honest assessment of the technical boundaries and failure modes of this model:

1. High-Cardinality Choice Questions (>20 Options):

   • When a choice question contains more than 20–25 options simultaneously, the token sequence expands and cross-attentive softmax over delimiter tokens diffuses.
   • Mitigation: Use ev.predict_shortlist() with bi-encoder cosine similarity filtering before cross-attentive scoring.

2. Non-Generative by Design (Prefill-Only System 1):

   • This model has no autoregressive decoder head (lm_head). It cannot generate conversational text, code, explanations, or summaries.
   • Mitigation: Pair ev with a generative System 2 model (e.g. Claude 3.5 Sonnet, GPT-4o, or DeepSeek R1). Use ev to filter, route, and gate requests in <30ms, escalating ambiguous or complex cases to the generative LLM.

3. Domain Specialization Without Fine-Tuning:

   • The model possesses strong general reasoning for enterprise workflows (triage, security alerts, invoices, moderation). However, deeply specialized domains (such as biochemical drug assays or complex statutory tax law) benefit from LoRA fine-tuning on domain data.

4. Extreme Context Lengths (>8,192 Tokens):

   • Documents exceeding 8,192 tokens experience quadratic memory growth and subtle attention dilution across large distances.
   • Mitigation: Pass summarized state or chunked segments, leveraging ev.cache prefix caching.

5. Subtle Code-Switching & Mixed Scripts:

   • Script detection uses character distribution thresholds. Subtle code-switching (e.g., predominantly English text with isolated foreign colloquialisms) may occasionally route to English unless explicit lang='multilingual' is specified.

📄 License & Attribution

• License: Apache 2.0
• Encoder Backbone: ModernBERT-large by Answer.ai & LightOn
• Repository: https://github.com/jaswanthsanjay88/rev