autotrust/JEV-35B

🔴 Live: AutoTrust/GuruSearch recommendation & search demo → news.guru.so

[!IMPORTANT] Try AutoTrust/GuruSearch live (new, 11 October 2026). A recommendation and search demo in which every ranking is a calibrated System 1 decision, made in about half a second:

• Recommendation: the latest headlines from 6 news feeds, ranked by importance.
• Search: results from Google News, Bing News and Yahoo News re-ranked by relevance, side by side with the search engines' own order, plus a short answer with citations.

A System 1 decision model on Qwen3.5-35B-A3B (35 B parameters, about 3 B active per token), up to 256 options in one pass

Decision Index 0.3, public suite: 59.73 (our scoring with the kit), against 53.64 for autotrust/JEV-27B-VL on the board. Approximate public vision score 71.94 (our rebuild of the vision benchmarks; JEV-27B-VL 71.67 on the same rebuild, a tie within noise). Median single-request latency 204 ms on one B200 (JEV-27B-VL 271 ms).

[!TIP] NVFP4 version: autotrust/JEV-35B-NVFP4, the same model with its routed experts in NVFP4: 25.6 GB download, 23 GiB of GPU memory (bf16: 72 GB, 66 GiB), so it runs on one 32–48 GB GPU. Decision Index 0.3 public 59.18 (bf16 59.73), vision rebuild 71.23 (bf16 71.94), 96.6 % of the Decision Index answers identical to bf16, same latency, same serve.sh and API.

Two models, two organisations. TypeSafe Jev 1.13 is the hosted, closed model made by TypeSafe AI. autotrust/JEV-35B is an independent open-weights model built by AutoTrust AI; it is not affiliated with, endorsed by, or a product of TypeSafe AI.

Decision Index 0.3 (public suite)

JEV-35B: all 140,178 scoreable requests of the 0.3 public suite answered (0 errors), System 1 only (thinking off), every choice read in one pass (up to 256 options), scored with the kit's score --edition 0.3. Our scoring, not a board entry: the board's Full score also counts private tests (80 %), which only its maintainers run. JEV-27B-VL: the board's public numbers for JEV-27B, whose text decisions JEV-27B-VL reproduces (see the JEV-27B-VL card).

JEV-35B is ahead in all five areas and on 24 of 37 benchmarks. Largest gains (skill points): PhishNChips +42.2, HoVer +28.4, Habermas Machine +28.1, VAST +22.7, WinoGrande +18.9, BANKING77 +18.3, iSarcasmEval +17.2, When2Call +14.9, GPQA +9.5. JEV-27B-VL is ahead on POP909-CL (+25.8), GSM8K (+18.2), ANLI (+11.4), BPoMP (+10.9), NLI4CT (+6.8) and BBH (+4.7).

Every benchmark

Skill rescales the benchmark's own metric so that chance is 0 (below chance counts as 0); ★ = gold benchmark (weight 1.2); the higher score is in bold.

Knowledge & Reasoning (area skill 0.426; JEV-27B-VL 0.414)

Language Understanding (area skill 0.632; JEV-27B-VL 0.564)

Retrieval & Classification (area skill 0.710; JEV-27B-VL 0.537)

Tools & Automation (area skill 0.764; JEV-27B-VL 0.735)

Arts & Human Taste (area skill 0.422; JEV-27B-VL 0.416)

Images (approximate public vision score)

The vision benchmarks of the Decision Index are not published; this is our rebuild of their public datasets (CV-Bench, BLINK, RealWorldQA, CharXiv, InfographicVQA, Mind2Web, CORD + FUNSD, Hateful Memes, R-Bench-M, MMMU-Pro vision; Winoground not included), scored with the board's chance correction and weights. The same rebuild gives JEV-27B-VL 71.67 against its board score of 71.53 on the same benchmarks.

Overall the two models are level: the difference (+0.3) is inside the noise (paired bootstrap 95% interval −0.9 to +1.4). The only statistically significant per-benchmark difference (paired McNemar test, p < 0.01) is moderation, in favour of JEV-35B (+10.4); MMMU-Pro (+3.1, p = 0.03) also leans to JEV-35B. The small deficits on RealWorldQA, R-Bench-M, CharXiv, InfographicVQA, CV-Bench and BLINK (−1 to −4) are not significant. The vision tower is Qwen3.5-35B-A3B's; image decisions are zero-shot.

Speed

One sequential client, the same 587 rows (387 text rows sampled across the Decision Index, 200 image rows), POST /v1/decide on serve.sh (vLLM), one B200:

Computer use, robot arm and games

Same demo code, seeds, scenes and opponents as the JEV-27B-VL card; every step is one System 1 decision (POST /v1/decide, thinking off), one model on one B200.

Computer use: screenshot → which element to click. A real browser (headless Chromium). Every clickable element gets a numbered box; System 1 picks the next click (or "the task is complete"), the browser clicks it, and the loop repeats.

Robot arm: pick and place from a camera image (MuJoCo). At every step System 1 answers two questions from the top camera: is the target left or right of the gripper, and above or below it? The arm halves its step whenever an answer flips.

• Computer use with element text: 95%, the same three failures as JEV-27B-VL (shop seeds 12, 15, 20: the colour swatch carries no text and is skipped).
• Numbered boxes only (every element read from pixels): 38% against 10%. JEV-35B completes 90% of the settings tasks but only 15% of mail and 10% of shop; most failures still declare the task complete too early (24 of 37).
• Robot arm: same success rate, different scenes. Each model misses 5 of 20 grasps (both miss scenes 4 and 19), every miss 3 cm or more off the cube centre; once grasped, every cube reaches the tray.
• Choosing directly among 8 motor commands fails for both models; decompose control into simple visual questions.

Games (seed 0 for both models; board as image + text):

For game play, use JEV-27B-VL. Over five seeds JEV-35B's largest 2048 tile averages 64 (random play: 102), it passes no Flappy Bird pipe and eats 3–18 pieces of food in Snake (mean 10). Per-episode results: reports/demos/.

Quick start (vLLM)

hf download autotrust/JEV-35B --local-dir JEV-35B
bash JEV-35B/serve.sh          # vLLM on :8000; one GPU with 80 GB or more

On a smaller GPU, use autotrust/JEV-35B-NVFP4 (one GPU with 32 GB or more; same serve.sh and API):

hf download autotrust/JEV-35B-NVFP4 --local-dir JEV-35B-NVFP4
bash JEV-35B-NVFP4/serve.sh

serve.sh runs serve_decide.py: the standard vLLM OpenAI server (same flags as vllm serve) with a POST /v1/decide route. The decision model is merged into the weights, so vLLM loads one plain checkpoint, with no LoRA. Tested with a vLLM development build from September 2026.

System 1: POST /v1/decide

curl localhost:8000/v1/decide -H 'Content-Type: application/json' -d '{
  "kind": "choice",
  "state": "Customer message: my card was charged twice for the same order.",
  "question": "Which team should handle this ticket?",
  "options": ["billing", "shipping", "technical support", "account security"]}'

The response has options, probabilities, choice, choice_index and usage. GET /v1/decide/info lists the defaults.

If you call vLLM's /v1/completions yourself instead of /v1/decide, pass top_k: 0 and top_p: 1.0 so that the returned probabilities are not truncated, then add the head bias and apply the temperature as below. JEV-35B is a decision model only: it does not chat or generate text.

Engine and read-out

System 1 reads the hidden state at the last token of a bare-text prompt:

[kind] choice
[state] ...
[question] ...
[options]
A) ...
B) ...
[decision]:

A 264-slot decision head gives one logit per slot; the active slots of the question's kind are soft-maxed with a per-kind temperature (calibration.json). The head is merged into the lm_head rows of the slot tokens (A–JU, false/true, 0–5); serve_decide.py adds the head bias (adapter_vllm/decision_head.json) and the temperature.

Limitations

• A System 1 decision model only: no chat, no text generation, no thinking mode.
• Knowledge & Reasoning is the weak area (GPQA 0.360, MMLU-Pro 0.642, HLE below chance); JEV-27B-VL is better on GSM8K, ANLI and BBH.
• Weaker than JEV-27B-VL at sequential game play (2048, Flappy Bird, Snake).
• The vision score is our rebuild, not the board's; image decisions are zero-shot.
• English-centric; not for high-stakes decisions without confidence gating.

Files

model.safetensors-* · config.json · tokenizer* · chat_template.jinja · generation_config.json · *_config.json
                          the decision model: Qwen3.5-35B-A3B architecture, decision head in the lm_head rows of the slot tokens
judge_config.json         slot layout, verbalizer ids, read-out
calibration.json          per-kind temperatures
adapter_vllm/decision_head.json
                          head bias and slot layout, applied by serve_decide.py
serve_decide.py · serve.sh
                          vLLM server with POST /v1/decide
videos/                   computer-use and robot-arm episodes
reports/demos/            per-episode computer-use, robot-arm and game results

NVFP4 weights (routed experts in NVFP4): autotrust/JEV-35B-NVFP4.

License

Apache-2.0. A fine-tune of Qwen/Qwen3.5-35B-A3B (Apache-2.0).