rsi-jev-v6.0-vl-4b

A typed-decision model built by a self-improving loop of AI agents. It reads a decision at layers 16, 20 and 32 of Qwen/Qwen3.5-4B-Base and answers each question at the first depth that is confident enough: easy questions stop early, hard ones use every layer. It answers choice / noul / score questions about a document and up to four images, and speaks the Jev HTTP API.

v6.0-VL is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (16, 20 or 32), not how many tokens it writes; it writes none.

pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
rsi-jev serve v6.0-vl-4b

Everything below is the version record from the project repository.

RSI-Jev v6.0-VL 4B

Describes release v6.0-VL · updated 2026-10-06

Each question picks its own depth. v6.0-VL reads a decision at three depths of one model — after layer 16, 20 and 32 of Qwen3.5-4B-Base — and answers at the first one that is confident enough. Easy questions stop early; hard ones go to the last layer. It takes the same typed questions as before — yes/no, pick-one-of-k, rate-on-a-rubric, about text and up to four images — and returns a calibrated probability for every option, plus the depth it answered at.

v6.0-VL is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (16, 20 or 32), not how many tokens it writes; it writes none.

It is a 4B model, named as Qwen names the 4.66B Qwen3.5-4B-Base: 4.69B parameters — the 32 decoder layers (3.57B), the token embeddings (0.64B), the vision tower (0.33B), the main decision head (0.05B) and the two early-exit heads (0.10B). The checkpoint is self-contained.

Checkpoint: shgao/rsi-jev-v6.0-vl-4b · this record: versions/v6.0-vl.md

1. The task

Unchanged from v5.0-VL: a state (text, optionally images), one or more typed questions, a probability for every option in one forward pass. New in the response: usage.depth, the layer each question was answered at.

2. How the model is built and trained

Full fine-tuning throughout (no LoRA), from Qwen/Qwen3.5-4B-Base, seed 17 unless noted, own-token option readout (specs in data/v6.0-vl_recipe/).

1. Trunk, main head and early-exit heads, 10,000 steps (batch 16 × 8 accumulation). A text corpus of 148 sources (453k training rows) with duplicates, label clashes and class-skewed sources removed. The early-exit heads at layers 12, 16 and 20 read a detached copy of their layer, so no gradient from them reaches the shared layers; the bottom 8 layers train at a tenth of the learning rate.
2. Early-exit heads refit, 3,000 steps, trunk and main head frozen, same corpus.
3. Images, 3,000 steps (batch 16 × 8): new image and text questions plus replay, 48,000 questions.
4. All four decision heads retrained under the own-token readout, 600 steps (batch 8, lr 1e-5, seed 1, trunk frozen).
5. Per-exit temperatures fitted on a held-out development set (no training), and the exit policy selected on it (below). The layer-12 exit lost to the cascade from 16 in every comparison and is not in the package.

Two findings shaped the recipe: retuning heads on a trunk trained with attached early exits did not recover the deep layers' accuracy, so the early exits are detached; and 10,000 steps on the cleaned corpus beat 40,000 on the older one on the held-out set (+0.026) and MMLU-Pro (+0.043).

Image sources and their licences. Stage 3 uses v5.0-VL's image corpus: the image sources of v4.0-VL and four sets generated by code. The images are the original datasets', under their own terms. Five sources are non-commercial or research-only, and several others carry no clear grant for their images:

The weights are released under Apache-2.0, as earlier releases were. Whether weights trained on non-commercial data inherit those terms is not settled; check the sources above against your use.

Exits. Decision heads read layers 16 and 20 as well as the last layer (32). Each exit has its own scalar temperature, fitted on held-out DEV rows (T = 1.88 at layer 16, 1.97 at layer 20, 1.65 at layer 32; the layer-32 value is the one in calibration.safetensors).

Choosing the exit. With auto, a question answers at the first exit whose calibrated top probability reaches that exit's threshold, 0.95 at layer 16 and 0.59 at layer 20, else at layer 32. With no effort set, one threshold, τ = 0.59, applies at both exits (multi-question requests). The thresholds were picked on one half of a held-out development set weighted like our test suite and confirmed on the other half, then read once on test; nothing was tuned on test or Decision Index rows. Mean depth on the suite: 23.3 with auto (20.9 with the single threshold).

3. Checkpoints

pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
rsi-jev serve v6.0-vl-4b                    # default: single questions at 32, auto on multi-question requests
rsi-jev serve v6.0-vl-4b --effort medium  # a server default; a request's own "effort" field wins

¹ Without open_jev_ood (see §8); the earlier rows include it. ² v6.0-VL's suite calibration is not compared with earlier releases' (§4.1); its final-set ECE is 0.036 as served.

The package was checked against the training checkpoint: in fp32 the package matches the training checkpoint at every exit (logits within 0.021, the same top answer on every question), and on 1,000 held-out questions every question takes the same exit as in training. Served in bf16 (the default on GPU), 1–2% of questions take a different exit than in fp32; accuracy on those rows is unchanged.

4. Results

4.1 Against v5.0-VL

Four ways to run the same weights, set per server (--effort) or per request ("effort" in the body): low stops at layer 16, medium at 20, high at 32, and auto answers at the first exit that is confident enough, with its own threshold at each exit (0.95 at layer 16, 0.59 at layer 20). With no effort set, single questions run at layer 32 and multi-question requests use a cascade with one threshold, 0.59, at both exits. Images always run all 32 layers. Suite is the fifteen-benchmark mean without open_jev_ood (see §8).

Default and auto are the bf16 package as served (auto: one pre-registered test read); low, medium and high are our fp32 reads of the same weights (text in fp32). Images are slightly below v5.0-VL (0.825 vs 0.829), within the release tolerance. Calibration is compared on the held-out final set only (0.036 as served by default and 0.024 with auto, against 0.042). The suite's calibration error is not compared with v5.0-VL's: it was not computed the same way for both.

³ v5.0-VL scored by its own evaluation script on the same item files. With auto, 20% of suite questions stop at layer 16, 46% at 20 and 34% go to 32: 23.3 layers on average (24.0 on the held-out set).

Which tasks need which depth (Decision Index sample, skill by exit; differences under 0.03 are noise):

• Done by layer 16: function calling (BFCL), intent (CLINC150, BANKING77), When2Call, sentiment (FinEntity), phishing, retrieval (BRIGHT, Amazon ESCI).
• Need about 20: commonsense and inference (HellaSwag, ANLI, BBH, NLI4CT), claim verification (HoVer), causal reasoning (CLadder), stance (VAST).
• Keep improving to 32: GPQA Diamond (0.136 / 0.177 / 0.218), multi-step reasoning (MuSR), code execution (CRUXEval), tool documentation (API-Bank), music (POP909).
• A few simple classification tasks get worse with depth: BANKING77 (−0.035), New Yorker caption matching (−0.060), Amazon ESCI (−0.037).

4.2 External: Decision Index 0.2.1

Full run, as served by default (every request of the suite, all 150,759 answered, nothing cut, the unset effort): Decision Index 0.2.1 46.24 (knowledge 28.8, language 46.2, retrieval 55.5, tools 65.8, arts 37.1), with the rows that also appear in our training data left out (46.25 with them). On the public board of 2026-09-28 that is the highest score among 4B models and anything smaller (14th of 71 overall). Nearby entries: JPT-4B 43.04 (the next of our size), Jet v6.2 42.60, Decider 4B 40.70, Intern-Decision-4B 37.81; for context, JPT-9B 46.89 and Lux 9B 43.49. v5.0-VL's full run scored 38.38.

By effort level (a 16k-question stratified sample of the suite, the kit's own sampler, the same rows for every level; a sample read, not a full run):

On this index, stopping at layer 20 scores the same as all 32. An earlier auto with one threshold of 0.59 at both exits read 44.89 here, 1.0 below both: it stopped 78% of questions at layer 16, and most of the loss was on two-option judgement tasks, where 0.59 is barely above chance. That is why auto now has a threshold per exit: 0.95 at layer 16, 0.59 at 20, chosen on our development sets (picked on one half, confirmed on the other, depth capped at 24 layers on average), then read once on test. On this sample the new auto reads 45.81 at 22.1 layers on average (15% of questions stop at layer 16, 62% at 20, 23% at 32). medium ties high on this sample but is not a better default in general. A pre-registered check on fresh held-out sets, read once, found layer 20 ahead of auto on Decision Index-like questions (+0.107), level with layer 32 there within noise (−0.018, interval −0.069 to +0.036), and behind auto on questions like our suite (0.711 vs 0.756). No single effort wins every workload, so the default stays as verified and the effort level is the user's choice. Leaving out the questions that also appear in our training data moves these numbers by at most 0.02.

Choosing a single threshold (confidence_threshold as one number applies it at both exits; range (0, 1]; an object such as {"16": 0.95, "20": 0.59} sets each exit, and that is auto's default). Dev sets, descriptive, not used for any selection: a suite-like dev set, a held-out dev set, and a dev set rendered like the Decision Index.

On suite-like and held-out text, every threshold from 0.59 up matches layer 32 within about 0.005, and 0.59 does so at about 21 layers. On questions rendered unlike the training data, a low threshold costs quality (0.535 vs 0.555); it comes back only from about 0.9 (26–28 layers). Layer 20 is within about 0.005 of layer 32 on all three. If inputs may be unusual, use a threshold of 0.9 or more, or medium.

4.3 Calibration

Each exit has its own scalar temperature, fitted so that confidence matches accuracy at that exit: final-set ECE after calibration 0.036 as served by default, 0.024 with auto (v5.0-VL: 0.042).

4.4 Which tasks and questions need depth

What the deeper layers buy on Decision Index 0.2.1, measured on the released package (bf16, one H200) on the same 16,000-row sample as the effort table in 4.2, with the 57 rows that overlap our training data removed. Fixed exits were forced with variants that share the released weights. A plain-language walk-through of these results is in docs/effort-depth.md.

By benchmark.

Skill at layer 16 / 20 / 32. About 315 questions per benchmark in the sample (standard error about 0.03), so only gaps of 0.03 or more count.

• Done at layer 16: BFCL function calling (.93 / .94 / .93), CLINC150 and BANKING77 intent, When2Call, FinEntity, PhishNChips, BRIGHT and Amazon ESCI retrieval, New Yorker caption matching.
• Needs layer 20: HellaSwag, ANLI, NLI4CT, BBH fixed-option (.39 / .48 / .47), HoVer claim verification (.21 / .32 / .32), CLadder, VAST, BPoMP (.53 / .75 / .76), ToolRet, the home appliance simulator.
• Keeps gaining to layer 32: API-Bank tool documentation (.60 / .62 / .66), GPQA Diamond (.14 / .18 / .22), MuSR (.34 / .36 / .39), CRUXEval (.07 / .10 / .13), POP909 (.52 / .59 / .62).

Smaller gains from 20 to 32 show on GSM8K, WinoGrande and Habermas. BANKING77 (−.035), New Yorker (−.060) and ESCI (−.037) get worse with depth. HLE, RAGTruth and iSarcasm score zero at every exit.

By question.

Per-question correctness at layers 16, 20 and 32, on the 27 benchmarks that Decision Index scores by per-question accuracy (10,232 questions; our per-question accuracy matches the kit's own score within 0.02 at every exit). Ranking, F1 and case-level benchmarks are covered only in the section above, because per-question accuracy does not measure them. Rescued is wrong at 16 and right at 32; harmed is the reverse.

By layer 16's own confidence (calibrated top-option probability):

By question shape:

Most of the gain happens between layers 16 and 20 (.652 → .686). From 20 to 32 the average moves by .004, which hides the knowledge and multi-step reasoning benchmarks above that keep gaining.

Choosing an effort level.

• low for intent, routing, retrieval and sentiment, where layer 16 is already saturated (57% of full latency).
• medium for classification and judgement workloads: the same Decision Index score as high at about 70% of the latency.
• high for knowledge, multi-step reasoning, code and tool documentation.
• auto for mixed workloads: its default thresholds (0.95 at 16, 0.59 at 20) match high on our suite at about 23 layers. Every response reports each question's exit (usage.depth) and calibrated confidence (usage.confidence), so the routing can be checked.

Limits of this analysis.

• 16,000-row sample reads, not full runs. This page compares exits with each other on the same rows, which a sample-to-full offset does not change.
• The per-question section uses the Decision Index test sample. It explains the released behaviour; it was not used to choose any setting.
• One seed, one read. Per-benchmark differences under 0.03 are within noise.

5. Speed and serving

Median latency per request on the Decision Index sample, one NVIDIA H200, bf16:

auto was read on a different GPU; at 22 layers on average it sits between medium and high.

medium matches high on the Decision Index at about 70% of the latency; low takes 57%.

Inputs up to 32,768 tokens, nothing cut (longer gets a 422); up to 5,120 options.

The API returns a probability and a confidence, and they are different numbers: confidence is the peak statistic Jev documents, (K * p_max - 1) / (K - 1). Route on the probability of the option you intend to act on. The response also carries usage.depth, the layer each question was answered at.

6. Limitations

• One seed for the head stage, as for v5.0-VL.
• The early exits read text only; questions with images always run all 32 layers.
• auto's thresholds were chosen on our development sets; on workloads unlike them, check usage.depth and usage.confidence in the responses, or set an effort.

7. How it got here

97 arms ran on this line: one per distinct experiment id with a run record, first recorded from 2026-10-03 through the run that chose auto's thresholds, a rerun counted once; 93 if the four data audits are left out. This window is counted by a different rule than the earlier sections, so the totals are not strictly comparable.

• Depth was lost in training, not in the head. On the first multi-exit model, retuned heads on the frozen trunk still left layer 32 below layer 20 on MMLU-Pro. Detaching the early-exit heads from the trunk gave a model that is no worse at layer 20 and better at 32.
• Shorter, cleaner training beat longer training on held-out data. 10,000 steps on the cleaned corpus beat 40,000 on the older one on the held-out set (+0.026) and MMLU-Pro (+0.043), and lost only on the in-distribution suite. Dropping rows because they were easy lost at equal steps; dropping only bad and class-skewed rows did not.
• The exit policy has to be fitted on data that looks like the test mix. Cascades tuned on in-distribution held-out rows stopped too early on hard questions. A development set of held-out proxies weighted like the evaluation mix, with one temperature per exit, passed every release bar.

The full list of arms on the released line is in EXPLORE.md.

Rerunning it

The training code, the stage specs and the commands are in data/v6.0-vl_recipe/; D is the data directory. The training corpora and the policy development sets are not public (listed in the recipe README), so the stages cannot be rerun from this repository alone. Serving the released model needs only the public code.

# stage 1: trunk, main head and early exits at 12 / 16 / 20 reading detached layers
python scripts/release_train.py --model Qwen/Qwen3.5-4B-Base --seed 17 --root D \
  --spec data/v6.0-vl_recipe/1-trunk-exits.json --corpus D/text-corpus --save-dir D/trunk/s17

# stage 2: the early-exit heads refit on the frozen trunk, distilled from the main exit at 0.5
python scripts/refit_exit_heads.py --ckpt D/trunk/s17 --corpus D/text-corpus --out D/refit \
  --exclude-public --exclude D/eval_suite_v2 --exclude D/eval_final_v2 --link-into D/trunk-refit/s17

# stage 3: images, from stage 2
python scripts/release_train.py --model Qwen/Qwen3.5-4B-Base --seed 17 --root D \
  --spec data/v6.0-vl_recipe/3-images.json --corpus D/image-corpus --save-dir D/images/s17

# stage 4: all four heads under the own-token readout, tower frozen (seed 1 shipped)
RSIJEV_OPTION_POOL_OWN_TOKENS=1 python scripts/fit_head.py --ckpt D/images/s17 --corpus D/image-corpus \
  --dev-corpus D/text-corpus --out D/heads-s1 --heads all --sdpa-math --max-length 4096 --skip-prefix vis --seed 1

# stages 5-6: per-exit dumps, the policy and auto's thresholds, the package (full commands in the recipe README)
python scripts/dump_exits.py dev --ckpt D/heads-s1 --corpus D/image-corpus --out D/dumps/s1   # and eval / cases / vision
python scripts/select_exit_policy.py policy ...       # temperatures per exit, cascade 16 -> 20 -> 32 at 0.59
python scripts/select_exit_policy.py thresholds ...   # auto: 0.95 at 16, 0.59 at 20
python scripts/package_multiexit.py --ckpt D/heads-s1 --seed-metric D/policy/seed.json --seed s1 \
  --policy D/policy/cascade.json --drop-exit 12 --thresholds D/policy/auto.json --out D/package

8. Corrections to v5.0-VL's record

The v5.0-VL record is left as published; these are its errata.

• §4.3 "all 150,512 requests answered": that run was 150,759 requests, all answered (150,512 was an earlier test run's count). The index, 38.38, is unchanged.
• A contamination audit after release found about 1,000 items of the Decision Index kit's test rows in the training corpora (ANLI 274, RouterBench-GSM8K 90, ARC 5, and BRIGHT/ToolRet query text without labels: about 0.3% of the kit's rows). Re-scored without them, the index moves by at most 0.04 on our sample. The internal suite task open_jev_ood overlapped 579 training rows, so the 15-benchmark suite (0.764) is inflated by an unknown amount; from this release the suite is reported without it. MMLU-Pro, BBH and the held-out final set have no overlap.