Eikos-27B — open typed-decision model (System-1)

Eikos-27B answers typed decisions about a given state in a single forward pass, with calibrated probabilities over a bounded set of options. It does not generate text at decision time.

Try it in the browser: the Eikos-4B demo on Hugging Face Spaces runs the smaller model with the same readout (ZeroGPU).

• Question types: noul / boolean (yes/no with probability), choice (one of N options) and score (ordinal levels).
• Outputs: the full probability distribution plus a confidence value, so callers can abstain below a threshold.
• Focus: global finance, trading and trade finance.
  • Apply a stated rule, policy or rulebook to a case: order limits, margin, wash sales, price bands, payments limits, KYC/AML, lending policies, documentary credits, Incoterms® stage mapping, VAT/GST.
  • Entity-level financial sentiment, answer verification over financial tables, temporal and numeric checks.
• Scope limit: it applies rules to the facts it is given. It does not predict prices.
• Languages: trained in English and Portuguese. Spanish was held out entirely and is used as a zero-shot test.
• Runs anywhere: one self-contained checkpoint in the official Qwen layout, with no adapters, no external LLM and no API.
  • vLLM, with prefix caching for many questions over the same state and for agent sessions;
  • PyTorch / transformers;
  • Apple Silicon via MLX or MPS.
• Images (serve v1.3): screenshots, charts, documents and photos, read zero-shot through the base model's vision encoder, with the same one-pass readout (GPU builds with vLLM). See Images (serve v1.3).

Eikos (εἰκός, "the probable") comes in two sizes: Eikos-4B and Eikos-27B (this model), each with FP8 and INT4 builds for GPUs and MLX builds for Apple Silicon (4B: 8-bit and 4-bit; 27B: 4-bit). Training data: caiovicentino1/eikos-decisions. Code: https://github.com/caiovicentino/eikos.

Numbers below are from our own evaluation harness (see Evaluation). Official JevBench runs are requested, for Eikos-27B-FP8 (this model's FP8 build) and for Eikos-4B. The organizers run the models themselves.

How it works

1. Prompt. The state (evidence), the question (criterion) and the lettered options go into a structured prompt, following the SemIf format.

2. Readout. The model reads the next-token logits only over the option letters. A softmax with a fixed temperature gives the distribution. Options are labelled A–Z and then AA, AB, … (each label a single token): up to 160 options are read in the same forward pass (serve v1.2, max_one_pass in decision_config.json), more through a tournament whose final round is one pass.

3. Calibration. Temperature T = 1 ships by default (see Calibration).

4. Training. Distillation from a strong teacher (GLM-5.3-Flash, maximum reasoning effort) on:

   • generated and blindly labeled decision items;
   • exact programmatic items;
   • compositional rules;
   • long-context dossiers (6k–12k tokens).

   Training-only signals, all switched off at inference:

   • soft cross-entropy to the teacher's probabilities, with option-order permutation;
   • an auxiliary loss on the item's short rationale (written with the item, explaining the gold answer).

   Eikos-27B is a single LoRA fine-tune of Qwen3.8-27B (rank 64, one epoch) on the same data as Eikos-4B, with long-context dossiers up to 12k tokens and without the EN↔PT view objective.

Quick start

vLLM (production: batching + prefix cache)

Requires vLLM ≥ 0.30.0. Older builds return wrong answers when several long requests are batched together on this hybrid (Gated DeltaNet) architecture; we measured drops of 3–6 points on long, shared-document items with vLLM 0.11. With 0.30 the batched results match the PyTorch reference.

bash serve_vllm.sh <MODEL_DIR> 8001      # vLLM engine: letter readout + hybrid prefix cache (checks vLLM >= 0.30)
python serve.py --model <MODEL_DIR> --vllm-url http://127.0.0.1:8001 --port 8000   # TypeSafe-compatible API on :8000

serve_vllm.sh runs vllm serve with the flags the readout needs: --served-model-name decider, --enable-prefix-caching --mamba-cache-mode align, --logprobs-mode processed_logprobs --max-logprobs 600, --dtype bfloat16 and --max-model-len 16384 (change it with MAX_MODEL_LEN). It also sets --max-num-seqs 64 (change it with MAX_NUM_SEQS): each request in flight holds one hybrid cache block, and vLLM's default of 1024 needs more blocks than a 27B build leaves free on one 80–96 GB GPU.

HTTP API (TypeSafe-compatible)

curl -s localhost:8000/v1/systemone -d '{
  "state": "Order ticket #A-2231. Retail client, moderate risk. BUY 1,500 XYZ at market. Equity USD 48,000. Last price USD 41.20. Rule 4.2: a single order may not exceed 50% of equity without written supervisor approval. Approvals on file: none.",
  "questions": {
    "allowed": {"type": "noul", "instructions": "Under rule 4.2, can this order be executed as submitted?",
                "criteria": {"true": "complies with rule 4.2", "false": "breaches rule 4.2"}},
    "action":  {"type": "choice", "instructions": "What should the desk do?",
                "criteria": {"execute": "send as submitted", "request_approval": "hold and ask a supervisor",
                             "reduce_size": "cut the order to the allowed size", "reject": "refuse the order"}},
    "risk":    {"type": "score", "instructions": "How risky is this ticket for the firm?",
                "criteria": ["low", "moderate", "high", "critical"]}
  }}'

• One pass for the whole request: all questions are answered together, and the shared state is processed once through the prefix cache.
• Response per question:
  • noul: probability (of "yes"), value and confidence;
  • choice: choice, probabilities and confidence;
  • score: score, probabilities, expected and confidence.

Agent sessions (incremental state + cache)

POST /v1/sessions                 {"state": "..."}          -> {"session_id"}
POST /v1/sessions/<id>/append     {"text": "..."}           -> grows the state
POST /v1/sessions/<id>/systemone  {"questions": {...}}      -> answers over the current state
DELETE /v1/sessions/<id>

On a hybrid (linear-attention) model the state is cached at the end of the context and forked for every question. In our tests a 15-step agent session gave answers identical to sending the full request each time.

Local (Apple Silicon)

python mlx_decide.py <MODEL_DIR>       # MLX backend: same readout and calibration as PyTorch
python local_demo.py <MODEL_DIR> mps   # PyTorch MPS

Images (serve v1.3)

Eikos also decides about images: screenshots, charts, documents and photos. Our training was text only and left the base model's vision encoder untouched; serve.py v1.3 passes the images to it and reads the answer with the same one-pass letter readout, so every option still gets a probability and no text is generated. This needs the vLLM backend (serve_vllm.sh) and works with all six GPU builds; the MLX builds are text only.

import base64, requests

img = "data:image/png;base64," + base64.b64encode(open("screen.png", "rb").read()).decode()
r = requests.post("http://127.0.0.1:8000/v1/systemone", json={
    "state": "",
    "images": [img],
    "questions": {
        "done": {"type": "noul", "instructions": "The upload finished without errors."},
        "next": {"type": "choice", "instructions": "Which button saves the file?",
                 "criteria": {"save": "the Save button", "cancel": "the Cancel button", "none": "no such button"}},
    },
})
print(r.json()["answers"])

• Formats: "images" takes data URIs or base64 strings. multipart/form-data works too (the JSON in a request field and the images as image files), and so do image data URIs inside the state, in a string, a dict or list value, or chat-style content parts: they are taken out in order and replaced with [image N].
• Limits: up to 4 images per request; images above 3840×2160 pixels are scaled down; image URLs only with serve.py --allow-image-urls. A 4K screenshot is about 8,300 tokens, within the 16k window serve_vllm.sh sets by default (MAX_MODEL_LEN changes it).
• Every question in a request reads the same images. Sessions stay text only.

Zero-shot accuracy on public sets, used for evaluation only. GUI grounding is asked as "which part of the screen holds the element for this action?": a 3×3 grid described in words (no marks), or a labelled 4×4 grid drawn on the screenshot.

• Against the base model: in the same one-pass format, Eikos-27B-FP8 is better than Qwen3.8-27B (the bf16 base model) on all six sets (300 items each) and better calibrated: MME 90.0% vs 73.7%, MMStar 70.7% vs 65.3%, ScreenSpot-Pro 65.7% vs 59.0%. The text-only decision training carried over to images. This compares both models in this format, not the base model at its best with generated answers.
• Every GPU build on the same 300 items per set (the FP8 and INT4 builds keep the vision tower in bf16):

• Limits: these are multiple-choice and yes/no decisions about a visible screen or picture. Pointing at an exact element without a grid or marks, multi-step reasoning about a screen and very small text were not measured. Calibration on images was not fitted separately: ECE is 0.03–0.10 per set, and both sizes are too confident on text-rich images (SEED-Bench-2-Plus: the 27B is 84% confident on average for 74.5% accuracy).

Evaluation

All numbers are our own measurements with a single harness. We never trained on any evaluation set. Spanish, one task family (trade-offs) and one topic (healthcare administration) were held out of training entirely.

* generated by the same rule generator family used in training (different rules and cases); see notes.

How to read the table:

• JevBench: public items only, not the official leaderboard run.
• Jev: measured through its API on our batteries, not distilled from.
• Laya (charts only): run with its official package; its JevBench scores are within one item of its official leaderboard results.
• Charts: 7,140 items common to all four systems. The confidence chart shows, for every confidence threshold, how many decisions a system would take on its own and how often those decisions are wrong.
• WCB (central-bank stance): reported as balanced accuracy, because the classes are imbalanced.
• Rules suites: "rules / new domain / rulebooks" come from the same generator family used in training, so they measure learning of that generator. Transfer to rules the model never saw is measured by "Trade (unseen rules)" (Incoterms®, documentary-credit presentation, EU VAT) and by RuleArena.
  • "New domain" has two parts. 492 items come from a domain absent from training (insurance): 27B 95.1, 4B 91.5. The other 108 items use a rule combination held out of training (an exception over a business-day window): 27B 91.7, 4B 90.7. That combination appears in only 6 of the ~6,000 rule rows used in training.

Long context

The case file is hidden inside unrelated text at the start, middle or end of the prompt (80 public JevBench decisions per point; released checkpoints served with vLLM 0.30).

Release builds

Final evaluation of every published build, run on the exact files in these repos: 7 suites, 7,371 items, vLLM 0.30 with batching and prefix cache on (MLX builds: MLX's CUDA backend on Linux). Batched vLLM is not bit-for-bit deterministic across runs (1–2 items per suite can change), so the bf16 rows differ slightly from the Evaluation table above, which uses our reference harness. "≥0.90" = share of decisions the model would take on its own at ≥90% confidence, and the real error rate among them.

Release gate, fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.

• Quantization cost: no build loses more than 2 points on any metric of any suite, and calibration (ECE) stays within 0.01 of bf16 or better.
• Eikos-4B-INT4 keeps accuracy and calibration but changes 4.5% of answers, above the 3% the gate allows. Almost all of them (94%) are on items where bf16 itself was unsure (confidence below 0.7); on decisions bf16 takes with confidence ≥0.9, it gives the same answer on 100% of items. We publish it with this note.
• Mac builds (MLX): Eikos-4B-MLX-8bit passes the gate. Eikos-27B-MLX-4bit and Eikos-4B-MLX-4bit keep accuracy and calibration but change 3.4% and 7.5% of answers. Most of those changes (92% and 93%) are on items where bf16 itself was unsure; on decisions bf16 takes with confidence ≥0.9, they agree on 99.9% and 99.8% of items. We publish them with this note. The small JevBench-original split (72 items) moves by a few items between builds.

Calibration

• Why T = 1: our held-out calibration sets turned out easier than real hard data. Every temperature fitted on them was below 1 and made the model over-confident on hard items. Absent a realistic calibration set, we ship T = 1.
• Changing it: calib.json holds the temperature, and callers can re-fit it on their own data.

Parallelism and speed

• Same state, many questions: the hybrid prefix cache processes the state once.
  • 200 questions over a 3.3k-token state ran 21× faster with vLLM prefix caching than without it.
• Throughput: batched decisions reach ~51 decisions/s on a shared GPU, with results identical to PyTorch.
• Full GPU (RTX PRO 6000, vLLM, hybrid prefix cache), 200 questions over one state:
  • 4B: 161 decisions/s on a 751-token state and 197 decisions/s on a 3.3k-token state;
  • 27B: 44 decisions/s on a 3.3k-token state.
• Our own PyTorch cache path: 1,500 questions over one state in 8.7 s (4B).
• Apple M4 16 GB, MLX bf16: ~0.4–0.8 s per decision, and 3 questions over one state in ~1.1 s.

Limitations

• Single pass means no multi-step reasoning. Long chains of arithmetic across many rules are out of reach; on RuleArena (NBA salary-cap rulebooks, ~25k tokens) the model does not discriminate. An optional "verify" mode (a short reasoning budget before the letter) exists in the server; it is off by default and not part of any reported number.
• General world knowledge is bounded by model size. Both sizes trail large frontier systems on knowledge-heavy tasks such as MMLU-Pro.
• Monetary-policy stance (hawkish/dovish) is a known weakness of both sizes: balanced accuracy 38.4 (4B) and 44.5 (27B), against 58.6 for Jev.
• Context: trained on inputs up to 32k tokens (4B) or 12k (27B). The base supports longer inputs, but accuracy beyond the trained range degrades gradually.
• Not advice: it is not legal, tax or investment advice. It applies the rules it is given and does not know your jurisdiction's current law.

Training data and licenses

See NOTICE.

• Model and code: MIT for our contributions. The base model is Apache-2.0 (LICENSE-Qwen).
• Third-party data, with attribution: FinEntity (ODC-BY 1.0), TAT-QA (CC BY 4.0), GSM8K train (MIT).
• Prompt format: SemIf (MIT).
• Trademark: Incoterms® is a trademark of the ICC. No ICC or regulator text was used.

Citation

@misc{eikos2026,
  title  = {Eikos: open, calibrated, single-pass typed-decision models for finance and trading},
  author = {Caio Vicentino},
  year   = {2026},
  url    = {https://github.com/caiovicentino/eikos}
}