Standard One 8B

Updated weights (v2.2, 2026-10-04). If you downloaded this model before, download it again or pin revision="v2.2". Earlier versions stay available under the tags v1, v1.1 and v2.

Version: v2.2

Standard One scores a bounded set of answers for a supplied scenario and returns probabilities through POST /v1/systemone. It does not generate free-form response text. This repository contains the merged BF16 8B checkpoint and the server code.

In the reported served evaluations, 8B scores higher than 3B on the public standard and hard tiers; 3B has a lower median latency on the measured short-request profile. See Benchmarks for the measurement conditions and limitations.

Historical v2 chart: the figure combines results from different measurement paths, measured 24–26 September 2026. See Changes in v2.2 for this version.

At a glance

• Send a state and a bounded rubric to receive probabilities for the supplied labels: choice selects among labeled options, noul is yes/no, and score uses an ordinal scale. The endpoint scores the labels in one forward pass without decoding answer text.
• In the same-run offline comparison with its untuned base (measured on v2), 8B improves on four of six suites and declines on public easy (2.08 percentage points) and public hard (4.50 percentage points). These are not served-endpoint results.
• Probabilities are temperature-scaled; calibration (hard-tier ECE, distribution total-variation) was checked on v2 — see Benchmarks below.
• The shared training mixture covers English, Japanese, Chinese, Spanish, French, German, Portuguese, Russian, and a smaller Korean share. See the nine-language MASSIVE intent results in docs/public-classification-suites.md; performance varies by language. The retained Pixtral vision tower accepts image data URLs; no separate image-input decision benchmark is reported.
• Apache-2.0: base model, adapter, merged weights, and shared server code hosted in the 8B repository.

Quantized versions

Validation numbers are in each repository's card.

Quick start

Follow the setup in QUICKSTART.md first: on a CUDA-capable Linux host, clone this repository and create the SGLang and adapter virtual environments with uv. The commands below assume the working directory and installations from that guide. Run the engine and adapter in separate terminals.

Engine (stock SGLang 0.5.20):

CUDA_VISIBLE_DEVICES=0 SGLANG_VLM_CACHE_SIZE_MB=0 .venv-sglang/bin/python -m sglang.launch_server \
  --model-path ./StandardOne-8B --served-model-name standard-one-8b \
  --host 127.0.0.1 --port 30000 --tp-size 1 --model-impl sglang --dtype bfloat16 \
  --context-length 32768 --max-running-requests 32 --mem-fraction-static 0.8 \
  --chunked-prefill-size -1 --disable-radix-cache --mm-preprocess-cache-size-mb 0 \
  --model-config-parser hf --load-format safetensors

Adapter (jev-adapter, ships as server/ in this repository):

.venv-native/bin/jev-adapter --engine-url http://127.0.0.1:30000 --model standard-one-8b --alias jev-latest \
  --host 0.0.0.0 --port 30120 --max-concurrency 1 \
  --tokenizer-model mistralai/Ministral-3-8B-Instruct-2512-BF16 \
  --tokenizer-revision f6fae9795746f63c9be8344932f01275f3c63734 \
  --prompt-wording served --label-scheme upper --default-temperature 1.65

Prompt wording

jev-adapter can phrase a request in two ways. served is the adapter's default and the wording used for the v2.2 measurements (--default-temperature 1.65, labels A–Z, then AA, AB, …); native accepts at most 26 options per question. The table below is a v2 measurement on the same served endpoint (no system prompt), with temperatures fitted on v2.

The 10 suites: judge proxy, hard proxy, stated-distribution probability, realistic transfer set, MuSiQue (multiple choice), SQuAD 2.0 unanswerable questions, ContractNLI, PAWS-X (English), a held-out hard decision set and a consistency set. None of them is a JevBench tier, and no JevBench item was used to choose the wording or the temperatures. To use native with its v2-fitted temperatures, pass --prompt-wording native --native-system-prompt none --default-temperature 0.85 --temperature-by-type choice=0.85,noul=0.85,score=0.70.

Try it:

curl -s http://127.0.0.1:30120/v1/systemone -X POST -H 'content-type: application/json' -d '{
  "model": "jev-latest",
  "state": "Policy: refunds require a receipt and purchase within 30 days. A customer bought 12 days ago but has no receipt. Issue a refund.",
  "questions": {
    "decision": {
      "type": "noul",
      "instructions": "Under the stated policy, is the requested action permitted? Treat unproved required conditions as not satisfied.",
      "criteria": {"true": "Every required condition is established and no prohibition applies.", "false": "A condition is missing or a prohibition applies."}
    }
  }
}'

Response (measured on 4 October 2026 with these v2.2 weights and the server/ adapter flags above, on one AMD MI350X; numbers rounded, adapter_elapsed_ms from a warm repeat). The receipt is missing, so the action is not permitted and the probability of true is low:

{
  "model": "standard-one-8b",
  "answers": {"decision": {"type": "noul", "noul": 0.0572}},
  "usage": {"input_tokens": 111, "output_tokens": 0},
  "metadata": {
    "confidence_method": "1 - normalized_entropy",
    "temperature": 1.65,
    "evaluations": 1,
    "label_scheme": "upper",
    "adapter_elapsed_ms": 22.6
  }
}

More (client command, 3B variant, request format): see QUICKSTART.md.

Changes in v2.2

v2.2 continues training from v2.1 with additional decision data. Questions with more than 26 options now use the labels A–Z, then AA, AB, …; the adapter in server/ uses this order by default (--label-scheme upper). Both versions were measured the same way: merged BF16 weights through SGLang 0.5.20 and jev-adapter, served wording, one option order, accuracy of the most probable answer; measured 1–3 October 2026.

Decision Index 0.2.1 (balanced skill): 41.14, measured through the adapter in server/ (served wording, default temperature 1.65). The tables below are the v2 measurements with native wording and are not directly comparable with the table above.

Benchmarks

Served endpoint results (the v2 release configuration). Merged BF16 weights through SGLang 0.5.20 and jev-adapter, native wording, no system prompt, one option order, per-answer-type temperatures (choice 0.85, noul 0.85, score 0.70). The wording and the temperatures were chosen on non-JevBench data. Jev 1.13 was measured on the same items through its hosted endpoint; its probabilities are raw, with no temperature applied. These are our measurements, not official sealed-set JevBench scores.

Offline comparison with the untuned base. This separate transformers runner used native wording, no system prompt, one option order, T=1. The base and tuned checkpoint were scored by the same offline path; these numbers are indicative of the base-model change, not the served scores above.

Against the untuned base, four suites improve, and public easy falls by 2.08 percentage points and public hard falls by 4.50 percentage points on this offline run. Served and offline probabilities differ even on identical prompts, so use the served table for expected endpoint behavior. Hard-tier ECE at the served temperatures is 0.184 against Jev 1.13's 0.099 raw, and mean TV to the stated distributions is 0.108 at served T against Jev's 0.192 raw; full calibration table: docs/BENCHMARKS.md.

Speed — raw serial latency on one H200 with SGLang 0.5.20, using a 242-decision profile averaging about 280 input tokens per decision. The 25.8 ms figure is p50 for this profile, not a latency guarantee for other request lengths, concurrency or hardware. Qwen checkpoints are untuned and shown for speed only; no accuracy comparison is implied.

Throughput on one H200 (hard+standard mix, 1,322 tokens/request): 29.3k tok/s at concurrency 1, rising to 39.5k tok/s at concurrency 64 (≈40 decisions/s at concurrency 8 on the 280-token profile above).

On the public classification and decision suites (400 cases/suite, seed 13, served endpoints): AG News 84.2 %, typed decisions 71.1 %, MASSIVE intent mean 86.1 %, email spam 91.8 %, phishing 85.2 %. Full table, per-language and per-workflow breakdown: docs/BENCHMARKS.md and docs/public-classification-suites.md.

A JevBench v1.4.1 run has been requested; the sealed-set result is not yet available. Full report: docs/BENCHMARKS.md.

Model details

• Base model: mistralai/Ministral-3-8B-Instruct-2512-BF16, revision f6fae9795746f63c9be8344932f01275f3c63734 (Apache-2.0).
• Adapter: LoRA r=16, α=32, dropout 0, on q_proj k_proj v_proj o_proj gate_proj up_proj down_proj of the language-model projections only (vision tower and multimodal projector excluded), 44,564,480 trainable parameters, PEFT 0.21.0. Adapter file adapter_model.safetensors, 214,559,872 bytes, sha256 123ddd039f4053e82e8ca18d7c247691dc49cbcabb6e1b7d98077bb7c5c7446e.
• Merged BF16 checkpoint: merging the adapter into the base changed 238 tensors (293 unchanged), none outside the language-model projections, max absolute weight change 0.00211.
• Serving details: served wording (the adapter default), no system prompt, --default-temperature 1.65 for every answer type (the value used for the v2.2 measurements; the adapter itself defaults to 1.0; no temperature was refit for v2.2), labels A–Z, then AA, AB, … (--label-scheme upper, the default; up to 255 options); served model name standard-one-8b behind stock SGLang 0.5.20 via jev-adapter (POST /v1/systemone); single caller-supplied option order, no rotation ensemble; 32,768-token context.

Training data

Training data is synthetic and format-augmented decision data plus decision items converted from public datasets (listed below); the JevBench public tiers used only for evaluation carry MIT. Full per-cohort breakdown (row counts, what each covers, licence): docs/BENCHMARKS.md.

Public datasets used (train splits where the dataset has one; licence as stated by each dataset; labels come from the datasets, distractor options are generated by code):

Upstream ids and the cohort each one feeds: docs/BENCHMARKS.md.

An exact-text overlap audit of the v2 training mixture against the public JevBench tiers found 0 exact scenario matches and 181 exact instruction matches — rows in two adequacy-rubric cohorts whose entire instruction field, a generic 58-character adequacy question, is byte-identical to one public hard-tier instruction (0.03 % of that 520,754-row mixture). These rows are kept and disclosed here rather than regenerated, since the overlap is limited to one rubric question's wording and never touches a scenario or an answer.

Limitations

• Public hard tier: the served v2.2 8B score is 56.76 %, versus 72.07 % for Jev 1.13. In the separate offline base comparison (measured on v2), Standard One 8B scores 55.86 %, below the untuned base's 60.36 %.
• Served probabilities are temperature-scaled by a fixed default (1.65 in the commands above, not refit for v2.2); if you apply this model to a materially different question distribution, re-fitting that temperature is advisable rather than assuming this value transfers.
• Up to 255 options per question with served wording (labels A–Z, then AA, AB, …, each one token); v2.2 was evaluated with up to 151 options. native wording accepts at most 26.
• The sealed JevBench set has not been measured for this model.
• Served and offline probabilities can differ on identical prompts (mean total-variation ≈0.06 on the hard tier, measured on v2); served numbers are treated as authoritative.
• Korean is a small share of multilingual training relative to the other seven languages.
• The card reports text benchmarks; it does not establish decision accuracy on image inputs.
• Ten-way support triage (36 %) and RAG passage relevance (59 %) were weak zero-shot on an earlier version; fine-tune for those.

Licence

Adapter weights, merged weights, server code (server/) and this card: Apache-2.0. Base model mistralai/Ministral-3-8B-Instruct-2512 (and -BF16): Apache-2.0 per its Hugging Face model card, which adds that the model must not be used in a way that infringes, misappropriates, or otherwise violates any third party's rights. jev-adapter and SGLang: Apache-2.0. The JevBench harness and public tiers used for evaluation: MIT; other benchmark items keep their own upstream terms.

Citation

StandardThinking/StandardOne-8B (merged weights + server code) · StandardThinking/StandardOne-8B-LoRA (adapter + merge recipe).