JPT-9B

JPT-0.8B · JPT-4B · JPT-9B · JPT-35B-A3B · llm2jev

What is JPT-9B

JPT-9B is a fast, open decision model: give it a situation and typed questions, get a calibrated probability for every option from one forward pass. No generated explanation, no reasoning tokens — latency is one prefill.

The large sibling of JPT-4B: the same recipe and the same data.

It implements the typed-decision interface introduced by Jev from TypeSafe AI [1]: a caller sends a state plus questions, and each question is one of three types. JPT is an independent model, not derived from Jev and not trained on Jev outputs; it is an open alternative behind the same interface.

Built on Qwen/Qwen3.5-9B: a LoRA fine-tune merged into full weights. The vision tower is unchanged.

Probabilities use one temperature T = 1.087, fit once on a held-out split — never per benchmark.

❯❯ Benchmarks

❯ JevBench v1.4.0

JevBench [2] scores general typed decisions; JPT-9B reaches 0.853 public accuracy, level with JevK5 and just under Winnow-12B and Jev 1.13.0 — but below the smaller JPT-4B (0.879). v1.4.0 has 231 public items and 308 sealed items only the maintainer can run, so JPT-9B has no official v1.4 score yet.

Source: other rows from results/v1.4/jevbench-v1.4-results.json at jevbench commit 2fa63fa (2026-09-23). Ours measured with JevBench's CLI at 2fa63fa (SGLang 0.5.18, kirp/jpt-9b@7114b0c): 197/231, easy 48 · standard 68 · hard 81. Our own harness gets 198: on original-intent-02-0 two options tie exactly (0.4864 each) and the two tools break the tie differently. ¹ Not in the v1.4 results; its own card's number on the 231 public items. A same-prompt Qwen3.5-9B zero-shot run on the public items has not been done yet.

❯ Decision Index 0.2.1

Decision Index [3] 0.2.1 (2026-09-27) is the broadest test: the full frozen suite, 38 scored benchmarks in five areas, chance-corrected — JPT-9B scores 46.89, the best 9B on the board and 0.2 behind Decider 35B-A3B. Run through llm2jev over SGLang and submitted as apolinario/decision-index#7; 0.2.1 rescores the same run (42.73 under 0.2).

Source: live board data/index-v0.2.1.json (generated 2026-09-27 16:59 UTC); full run and scores.json in kirp/decision-index-results-jpt-9b (gated: it carries the suite's GPQA/HLE item text).

By area, against Jev 1.13.0 on the same items and scorer: JPT-9B is behind Jev in all five areas and ahead of it on 4 of the 38 index benchmarks.

Chance-corrected skill × 100 (0 = random, 100 = perfect). Jev's numbers are its official entry on the live board (jev-1.13.0); ours are from the same kit and suite.

❯ More benchmarks

Against JPT-4B on the same evals: JPT-9B is better on NLI, intent classification and EnvBench public, and worse on the JevBench hard tier. Same-prompt base-model (Qwen3.5-9B zero-shot) rows exist only for EnvBench; the image evals (ScreenSpot-v2, Screen2Words, ERQA) have not been run on this checkpoint.

Jev 1.13.0 has no official score on these splits (our own test/dev cuts, EnvBench, and the image sets), so its column is "—"; its official scores on the Decision Index versions of ANLI and BANKING77 are in the per-benchmark table above. Running the Jev API on these splits would fill them.

Qwen3.5-9B, same prompt, zero-shot on EnvBench v0.1: 24.2 public / 25.3 held-out. Banking77, MASSIVE, AG News, Emotion, SST-5 and typed rows are in-distribution (train splits in the mix, test items not).

❯❯ Quick Start

Two pieces: an engine that holds the weights, and llm2jev (>= 0.6.1) in front of it, reading option probabilities off the engine.

⚡ SGLang (recommended)

python -m sglang.launch_server --model-path kirp/jpt-9b --port 30000 \
  --context-length 32768 --mamba-scheduler-strategy extra_buffer &  # Qwen3.5's DeltaNet layers need this flag
llm2jev --model kirp/jpt-9b --backend sglang --url http://127.0.0.1:30000 --port 8080 --temperature 1.087

Tested with SGLang 0.5.9; install cuDNN 9.15+ over its pinned 9.10: pip install "sglang==0.5.9" && pip install "nvidia-cudnn-cu12>=9.15".

🔁 vLLM

vllm serve kirp/jpt-9b --max-logprobs 256 --return-tokens-as-token-ids --enable-scale-out --port 8000
llm2jev --model kirp/jpt-9b --backend vllm --url http://127.0.0.1:8000 --port 8080 --temperature 1.087

The three vLLM flags are required: without them every request is a bare HTTP 400.

🧪 No engine (quick check only)

pip install "llm2jev[hf,vision]"
llm2jev --model kirp/jpt-9b --backend hf --port 8080 --temperature 1.087

Serializes requests; for traffic use SGLang or vLLM.

📨 Ask it a question

import requests
r = requests.post("http://127.0.0.1:8080/v1/systemone", json={
    "state": "Refund policy: full refund within 30 days of purchase; 50% until day 60; none after.\n"
             "Order 1182 was bought on 3 March and returned on 20 April.",
    "questions": {
        "refund": {"type": "choice", "instructions": "What refund does order 1182 get?",
                   "criteria": {"full": "Full refund", "half": "50% refund", "none": "No refund"}},
        "late":   {"type": "noul", "instructions": "Was the return made after day 30?",
                   "criteria": {"true": "Yes", "false": "No"}}}})
print(r.json()["answers"])   # each answer has the per-option probabilities

🖼️ With images

state = [{"role": "user", "content": [
    {"type": "image", "image": "https://example.com/screen.png"},
    {"type": "text", "text": "Task: open the settings page. Numbered boxes mark clickable elements."}]}]
questions = {"click": {"type": "choice", "instructions": "Which box should be clicked?",
                       "criteria": {"1": None, "2": None, "3": None, "4": None, "5": None}}}

❯❯ Training

The JPT-4B recipe and data (mix_train_env_v11, 49,221 typed questions) on a larger base, one epoch, merged into full weights.

❯❯ Limitations

• Not strictly better than JPT-4B. The JevBench hard tier (0.730 vs 0.784) and the EnvBench held-out game area (0.288 vs 0.315) are lower, despite lower training and validation loss throughout. Not yet root-caused.
• Arithmetic and dates are its weakest area: it answers from the evidence given and has no reasoning phase by design.
• Up to 255 options are accepted; training covered up to 77 (Banking77).
• English first. Other languages come only from a few multilingual classification sets.
• Images are untested on this checkpoint; they go zero-shot through the base vision tower.

❯❯ References

1. TypeSafe AI. Jev. https://typesafe.ai
2. F. Standhartinger. JevBench, v1.4.0. https://github.com/fstandhartinger/jevbench
3. Decision Index, edition 0.2.1. https://huggingface.co/spaces/multimodalart/jev-decision-index
4. Nie et al. Adversarial NLI. ACL 2020.
5. Casanueva et al. Efficient Intent Detection with Dual Sentence Encoders (Banking77). NLP4ConvAI 2020.
6. FitzGerald et al. MASSIVE. ACL 2023.
7. Zhang et al. Character-level Convolutional Networks for Text Classification (AG News). NeurIPS 2015.
8. Saravia et al. CARER: Contextualized Affect Representations for Emotion Recognition. EMNLP 2018.
9. Socher et al. Recursive Deep Models for Semantic Compositionality (SST). EMNLP 2013.
10. EnvBench, v0.1 (ours, frozen 2026-09-23; not yet public): programmatically solved game, planning and rule decisions with exact gold answers.

❯❯ License

CC BY-NC 4.0. The weights derive from Qwen3.5-9B (Apache-2.0), but some training datasets allow only non-commercial or research use, so the model is released for non-commercial use.

JPT-9B is an independent open model that implements a typed-decision interface (noul, choice and score questions answered with probabilities). It is not affiliated with, endorsed by or derived from TypeSafe AI or its Jev model, and it was not trained on Jev outputs.