rsi-jev-v6.1-vl-27b

A typed-decision model built by a self-improving loop of AI agents. It is the weight average of two fine-tunes of Qwen/Qwen3.8-27B, with decision heads at layers 48, 56 and 64: each question answers at the first depth that is confident enough. It answers choice / noul / score questions about a document and up to four images, and speaks the Jev HTTP API.

v6.1-VL 27B is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (48, 56 or 64), not how many tokens it writes; it writes none.

It runs in bf16 on one 96 GB GPU.

pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
rsi-jev serve v6.1-vl-27b --effort auto      # recommended; or RSIJEV_EFFORT=auto

from rsijev import Decider

d = Decider("v6.1-vl-27b")
d.decide("I was charged twice. Please refund.",
         {"refund": {"type": "noul", "instructions": "Does the user request a refund?"}})

Everything below is the version record from the project repository.

RSI-Jev v6.1-VL 27B

Describes release v6.1-VL 27B · updated 2026-10-08

v6.1-VL on a 27B base. The same method as v6.1-VL 4B, on Qwen/Qwen3.8-27B: two fine-tunes of the same parent, averaged weight for weight, with decision heads at layers 48, 56 and 64. A question answers at the first head that is confident enough. The effort levels and the API are v6.1-VL's.

On the Decision Index 0.3 public set it scores 65.64 with --effort auto, the setting we recommend, and 65.46 with effort unset. The official full score needs a submission, which is pending. Against v6.1-VL 4B it is more accurate on every set we read, and calibration is a little worse (final ECE 0.064 against 0.048).

v6.1-VL 27B is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (48, 56 or 64), not how many tokens it writes; it writes none.

It is a 27B model: 26.66B parameters, all of which the package ships and runs — the 64 decoder layers (24.35B), the token embeddings (1.27B), the vision tower (0.46B), the main decision head (0.19B) and the two early-exit heads (0.38B). The checkpoint is self-contained. It runs in bf16 on one 96 GB GPU.

Checkpoint: shgao/rsi-jev-v6.1-vl-27b · this record: versions/v6.1-vl-27b.md

1. The task

Unchanged from v6.1-VL: a state (text, optionally up to four images), one or more typed questions, a probability for every option in one forward pass, and usage.depth, the layer each question was answered at.

2. How the model is built and trained

Base: Qwen/Qwen3.8-27B (revision 1d4bf0f2, Apache-2.0; a post-trained model, no Base variant exists). Every training stage is LoRA (rank 64, alpha 64) on attention and MLP projections of all 64 layers, with the embeddings frozen; each stage's adapter is merged into the bf16 tower before the next. The option readout is v6.1-VL's: cross-attention over each option's own tokens.

1. A shared parent, three stages from the base:
   • text, 10,000 steps of batch 16 on a 401,023-question corpus, inputs up to 16,384 tokens;
   • 2,000 steps of batch 64 on Decision Index training pools (half) and replay of the first stage's corpus (half);
   • 2,000 steps of batch 64 on a second such mix, which adds MMLU-Pro-style and other hard multiple-choice questions.
2. Two fine-tunes of that parent, trained apart on different mixes of the same kind (half Decision Index training pools, half replay; seed 17):
   • member A: 2,000 steps of batch 64, 128,000 training questions;
   • member B: 3,000 steps of batch 64, 192,000 training questions, from a mix with several pools rebuilt and six more replay sources left out, and with an exit head at layer 48 trained together with the trunk.
3. The average. Every tensor of the tower and the main head is the mean of the two members, weight 0.5 each, summed in fp32 and stored in bf16. Nothing is trained after. This is a uniform model soup (Wortsman et al. 2022), the same method as v6.1-VL 4B.
4. Exit heads at layers 48 and 56, trained for each member on its frozen trunk, then averaged the same way. On held-out development rows the average answers better at both exits than either member: 0.832 at layer 48 (members 0.825, 0.828) and 0.841 at layer 56 (0.835, 0.838).
5. Per-exit temperatures and the exit policy, fitted on development rows with v6.1-VL's rules (below). The 1,848 development rows that overlap the members' training mixes were dropped first.

How the trunk was chosen. The average was chosen from 9 full Decision Index 0.3 reads of candidate trunks, each answering at layer 64 with the same serving code on one H200. The candidates were the two members alone (64.83 and 65.30), their 0.5 / 0.5 average (65.59), uneven 0.3 / 0.7 and 0.4 / 0.6 averages (65.49, 65.57), three-way averages with their parent or a further fine-tune (65.37, 65.57, 65.58), and a further fine-tune of member B (65.01). The uniform average read highest; the next three were within 0.02, and no noise estimate separates them. The package was then read once more on the release serving path (§4.2).

Images. The package carries the base's vision tower unchanged. There is no separate image training stage: images go through the base's vision tower and the full 64 layers, and the decision heads read the result as they read text. Held-out image accuracy is 0.858 (eval_vision_v1), against 0.834 for v6.1-VL 4B, which had image training.

Training sources and their licences. The Decision Index training pools include non-commercial sources (ANLI, CC BY-NC 4.0; the RAGTruth QA part, MS MARCO terms; the RAGTruth data-to-text part, Yelp terms) and sources with no licence file (VAST, ACOS, NLI4CT), as well as share-alike HoVer and ForecastBench. The MMLU-Pro-style and hard multiple-choice questions come from the sources v6.1-VL 4B's record lists (among them SciQ, CC BY-NC 3.0), and were filtered against the 1,000 MMLU-Pro questions we evaluate on. The chess, POP909 and home-simulator questions have no licence recorded yet. The weights are released under Apache-2.0, as earlier releases were. Whether weights trained on non-commercial data inherit those terms is not settled; check the sources against your use.

Exits. Each exit has its own scalar temperature: T = 1.01 at layer 48, 1.02 at 56, 1.05 at 64. The layer-64 value is in both calibration.safetensors and calibration.json.

Choosing the exit. With auto, a question answers at the first exit whose calibrated top probability reaches that exit's threshold, 0.8 at layer 48 and 0.7 at layer 56, else at layer 64. These were picked by v6.1-VL's rule on one half of the development sets (24 of 48 settings feasible) and confirmed on the other, then read once on test. With no effort set, a one-question request answers at layer 64, and a multi-question request uses one threshold, 0.8, at both early exits. Mean depth on the suite with auto: 52.5 layers.

3. Checkpoints

pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
rsi-jev serve v6.1-vl-27b --effort auto      # recommended (or RSIJEV_EFFORT=auto); a request's own "effort" field wins
rsi-jev serve v6.1-vl-27b                    # effort unset: single questions at 64, the 0.8 cascade on multi-question requests

¹ Without open_jev_ood; the earlier rows include it. ² The suite's calibration is not compared across releases (v6.0-VL's record, §4.1); final-set ECE is in §4.1 below. The 27B figures are the bf16 package at layer 64, which is what a one-question request gets by default. The 4B figures are from v6.1-VL's record.

4. Results

4.1 By effort, and against v6.1-VL 4B

One read each on the same evaluation sets with the same aggregation; the 27B in bf16, ECE at the served temperatures. low, medium and high stop at layers 48, 56 and 64; auto uses the per-exit thresholds above. Images always run all 64 layers. Suite is the fifteen-benchmark mean without open_jev_ood. KoBBQ, CLINC150 and the images come from the release gates (§4.3).

auto matches layer 64 on the held-out set at 52.5 layers on average; MMLU-Pro drops 0.006. The early exits cost most on MMLU-Pro (−0.058 at layer 48), little elsewhere. Calibration is worse than v6.1-VL 4B's at every setting and inside the 4B's release bar of 0.078. The KoBBQ and CLINC150 columns are effort unset (high for one question) and auto.

4.2 External: Decision Index

Decision Index 0.3, public set: 65.64 with --effort auto, the setting we recommend and submit, and 65.46 with effort unset, 140,178 requests each, coverage 1.0, nothing cut, on the release package with the release serving code (one H200). With the rows that also appear in our training data left out, both are unchanged (65.64, 65.46). The official full score needs a submission, which is pending.

auto gains 1.6 on tools and loses 0.4 on knowledge, the area that needs the depth.

Decision Index 0.2.1, full run: 64.92 with auto and 64.72 with effort unset (all 150,759 requests answered, nothing cut), with the rows that also appear in our training data left out (64.92 and 64.71 with them). v6.1-VL 4B: 50.74.

The trunk alone, before packaging, read 65.59 on the 0.3 index; the package reads 65.46 with effort unset. The difference is the serving path (§5) and the refit heads and temperatures, inside the 0.3 drop the release gate allowed.

4.3 Release gates

Bars written before the release package was read: each is the stricter of v6.1-VL 4B's bar and a stand-in 27B checkpoint's own read minus about two standard errors. The Decision Index bar is a drop of at most 0.3 from the chosen trunk's read.

All pass. Effort high was also read on KoBBQ and the long inputs and passes at the same values as effort unset.

4.4 Calibration

Each exit has its own scalar temperature. Final-set ECE after calibration is 0.064 at layer 64 and 0.069 with auto, against v6.1-VL 4B's 0.048 and 0.055. If you route on usage.confidence, check it on your own data.

5. Speed and serving

One request at a time, bf16, single process. low and medium were not timed on these runs.

Padded serving. The package sets serving.pad_multiple = 64 in meta.json. Short text batches (rows × padded width ≤ 768 tokens) are right-padded to a multiple of 64 tokens and replayed as CUDA graphs; longer batches, prefix-cache plans and images run the unpadded path unchanged. Graphs on and off give the same bits. Short one-question requests run in about half the time (×0.45 under 128 tokens, ×0.54 at 128–256). The model's numbers above are defined on this path: against the unpadded path, 1,793 of 1,800 KoBBQ and CLINC150 answers agree, and the 7 that move are near-ties. RSIJEV_PAD_MULTIPLE=0 turns it off; packages without the setting, every earlier release included, serve as before.

Inputs up to 32,768 tokens, nothing cut (longer gets a 422); up to 5,120 options; own-token option pooling, set in meta.json.

The API returns a probability and a confidence, and they are different numbers: confidence is the peak statistic Jev documents, (K * p_max - 1) / (K - 1). Route on the probability of the option you intend to act on. The response also carries usage.depth, the layer each question was answered at.

6. Limitations

• The trunk was chosen from 9 full Decision Index 0.3 reads on the same benchmark it reports. The top five candidates read within 0.1 of each other, so the chosen one's lead is not separated from noise, and its score carries some selection on that benchmark. The package's own reads (65.64 with auto, 65.46 unset) are fresh runs, after the choice.
• Calibration is worse than v6.1-VL 4B's: final ECE 0.064 against 0.048, 0.069 against 0.055 with auto (§4.4).
• Images use the base's vision tower with no image training of our own. The 27B was not read on image tasks beyond eval_vision_v1.
• It needs one 96 GB GPU in bf16 (84.3 GB peak at the 32,768-token cap). There is no smaller build of it.
• Serve it as packaged: the numbers are defined on the padded path. The unpadded path (RSIJEV_PAD_MULTIPLE=0) moves about 0.4% of answers, all near-ties.
• low and medium were not run on the Decision Index; the RTX PRO 6000 was timed with effort unset only; auto exits earlier but was not timed there.
• The unset path's single threshold for multi-question requests, 0.8, was not chosen by a rule of its own; the Decision Index run with effort unset is its read.
• One seed per stage. The training corpora are not public.

7. How it got here

The 27B line trains with LoRA from a post-trained base, on the Decision Index training pools with replay, then averages two fine-tunes as v6.1-VL 4B did. What this release established:

• Averaging two fine-tunes beat both, at 27B too. Member A alone read 64.83 on the 0.3 index, member B 65.30, their average 65.59, the same pattern as v6.1-VL 4B.
• The averages have plateaued. Six averages of the same family read 65.37 to 65.59; uneven weights and a third ingredient did not move the score. A further fine-tune of member B read 65.01 alone, +2.4 on tools and −3.0 on language against the average.
• Exit heads average too. The average's heads at layers 48 and 56 are the mean of the members' heads, never trained on the averaged trunk, and they beat either member at both layers.
• Padding is what makes it fast at one request. Short requests on the 27B are bound by kernel launches; padding to a fixed width lets CUDA graphs replay them (§5).

Rerunning it

The 27B training stages (LoRA on four GPUs) and their corpora are not in this repository. Serving the released model needs only the public code.

8. Corrections to v6.1-VL's record

None. The v6.1-VL record is left as published.