Bobcat Flash 1.2

Technical write-up: foxl.ai/blog/bobcat-typed-decisions

Bobcat Flash 1.2 · Bobcat Flash 1.2 FP8 · Try it in the Space · Code on GitHub

Bobcat Flash is the fast tier of Bobcat, a typed-decision model. You send a state (text or JSON) and questions whose possible answers you define in the request; Flash returns a probability for every answer you named, and nothing else. It never generates text: it reads the logits of the offered candidates at the first answer position of one forward pass, and the host builds a closed JSON reply from your own names. An answer can be wrong, but it cannot be malformed.

This repository, sanghwa-na/bobcat-flash-1.2, holds Bobcat Flash 1.2 as ready-to-serve BF16 weights on google/gemma-4-26B-A4B-it (Apache-2.0) at revision 4d7ae4984b7db7de8f8457170b3f1a419ee76d52, a sparse mixture-of-experts model with about 4B active parameters per token. vLLM serves it in FP8, and it reads inputs of up to 98,368 tokens without truncating them. The compiler, servers and evaluation code are at github.com/foxl-ai/bobcat.

Highlights

Jev figures are TypeSafe's own published answers; Jev was never called. The same-base baseline is untrained Gemma 4 26B-A4B-it with the same compiler and readout. See Limitations for what this model does not do well.

What it does

The request and reply use the shapes of TypeSafe's published System One HTTP API, so the official typesafe-sdk works against a Flash server by changing its base URL. Inputs over the server's limits are refused with HTTP 422 and never truncated.

Quickstart: serve Flash on one GPU

A clean install with the commands below was tested on one RTX PRO 6000 Blackwell 96 GB with vLLM 0.30.0, before these weights were uploaded: a fresh Python 3.12 venv with exactly the install line below, then the server command below on the merged weights (the download line itself runs once the repository is public). The SDK example answered payments; TypeSafe's 20 workflow cases agreed with the reference on 90.6% of 329 questions, at a median 0.27 s per case over loopback HTTP. On the same GPU and server command, all 240 JevBench-format inputs of 61K to 96,161 tokens were answered (no failure, no refusal).

curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"   # uv
git clone https://github.com/foxl-ai/bobcat && cd bobcat
export UV_PYTHON_PREFERENCE=only-managed   # a uv-managed Python ships the headers Triton compiles against
uv venv --python 3.12 .venv-serve
uv pip install --no-config --python .venv-serve/bin/python vllm==0.30.0 fastapi uvicorn scipy jinja2 \
  "tokenizers>=0.21" huggingface_hub typesafe-sdk==0.7.1
export PYTHONPATH=$PWD/src PY=.venv-serve/bin/python

# These weights (about 52 GB), then the typed-decision server, FP8 at load, 96K states
$PY -c "from huggingface_hub import snapshot_download as s; s('sanghwa-na/bobcat-flash-1.2', local_dir='bobcat-flash-1.2')"
VLLM_USE_FLASHINFER_SAMPLER=0 $PY -m bobcat.flash_server --engine vllm --model bobcat-flash-1.2 \
  --compiler-model bobcat-flash-1.2/compiler --quantization fp8 \
  --temperature choice=0.594604,noul=0.353553,score=0.529732 \
  --name bobcat-flash-1.2 --release-date 2026-10-07 --max-num-seqs 64 --max-model-len 98368 \
  --schedule all --engine-arg max_num_batched_tokens=16384 --engine-arg attention_backend=TRITON_ATTN \
  --host 127.0.0.1 --port 8000 --local

Then call it with the official SDK:

import os
os.environ.update(TYPESAFE_BASE_URL="http://127.0.0.1:8000", TYPESAFE_API_KEY="local",
                  TYPESAFE_DEFAULT_MODEL="bobcat-flash-1.2")
from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient()
result = client.system_one(
    "I was charged twice for the same order. Can someone look into this?",
    {"billing": Noul(instructions="Is this about billing?"),
     "route": Choice(instructions="Which team should handle it?",
                     criteria={"payments": "Charges and refunds", "shipping": None,
                               "account": "Login and settings"})},
)
print(result.nouls["billing"].noul, result.choices["route"].choice)

Notes:

• --temperature choice=0.594604,noul=0.353553,score=0.529732 applies one calibration temperature per question type, fitted on a held-out calibration split of JevBench-format decisions (generated and public, sharing nothing with JevBench) for the best JevBench-style Capability. A temperature never changes an argmax. --temperature 0.9232 instead applies the single temperature fitted by log-loss on the held-out calibration split of the six product tasks; it gives better-calibrated probabilities on those tasks, and the sharper per-type values commit more often on yes/no questions.
• --quantization fp8 is vLLM's online dynamic FP8 of these BF16 weights, the served configuration.
• --max-model-len 98368 reads states up to about 96K tokens in full (state plus question); longer inputs are refused with HTTP 422 and never truncated. On a GPU with less memory, lower it; the limit only moves the refusal point.
• --local disables the shared secret the servers otherwise require; use it only on a loopback or private interface.
• --compiler-model bobcat-flash-1.2/compiler points the Bobcat compiler at the base model's pinned tokenizer, template and configuration; it checks them against the download receipt in that folder. Run the server from the repository root, or pass --identifiers bobcat-flash-1.2/bobcat-identifiers.json.

Running on AWS

These are ordinary GPU Linux hosts; the Quickstart above is the whole recipe.

• Amazon EC2. One RTX PRO 6000 Blackwell 96 GB (g7e.2xlarge) runs the clean install, the server at 98,368 tokens and every latency figure in the first column above; an H100 80 GB (p5.4xlarge) gave the second column. Use a Deep Learning AMI with a recent NVIDIA driver and allow about 80 GB of disk.
• Amazon SageMaker AI. The same commands run in a JupyterLab space or notebook instance of an equivalent GPU type. A real-time endpoint needs a custom container that runs bobcat.flash_server behind SageMaker's /invocations and /ping routes; we have not published or tested one.

Evaluation

Differences are paired, with 95% bootstrap intervals over source components. The development and sealed-final figures use the evaluation path (these merged BF16 weights, bobcat.flash_train --eval-only); TypeSafe, injection, long-state and JevBench-format figures use the weights served in FP8 over HTTP.

Sealed final

A fresh sealed split of the product evaluation (1,609 decisions, four tasks, Korean), opened once, after the release manifest that binds these weights, their temperatures and the analysis was frozen. Nothing was changed after it. Failures count as wrong (there were none).

• Paired, over source components: Flash 1.2 minus Flash 1.1 +0.80 [-0.09, +1.79]; minus Bobcat 1.1 -0.64 [-1.93, +0.45].
• Pre-declared sensitivity analysis, without the 50 rows whose text also appears in a training file (all from the earlier Flash training corpus; none from the data added for 1.2): 93.91%, +0.80 [-0.06, +1.70] over Flash 1.1.
• The release bar fixed before the opening was 92.21%; it was met.
• Served in FP8 (vLLM, engine path), the same decisions score 93.54%.
• Log-loss 0.361 and ECE 0.014 at the product temperature (Flash 1.1 0.397 / 0.009).

Six-task development evaluation

The product evaluation's development split (3,188 decisions, Korean), the same split the release gates used.

TypeSafe's published workflow examples

20 cases, 329 questions answered by every published system. Agreement with TypeSafe's reference answers: 90.27% (Jev 90.9%, published; same base zero-shot 89.67%). Mean of the four workflows' agreement: 83.7% (Jev 86.5%, published). The release server on an RTX PRO 6000 (the clean install above) agreed on 90.6%, and a second clean install from the public repositories on 89.97%; the gate value 90.27% is from the evaluation server. TypeSafe 20 (329 questions; one question = 0.30 pt) varies between fresh FP8 servers: 89.97–90.6% across runs. The gate run was 90.27%; the corrected Flash-line bar was 90.19%.

Wrong answers named inside the state

300 questions, each with three attacks that name a wrong answer inside the state (an administrator verdict, a sentence inside the text, a note to an AI grader). Attack success: 8.2% (same base zero-shot 31.1%). Accuracy: clean 91.7%, administrator verdict 88.3%, inside the text 81.7%, note to an AI grader 83.0%.

Long inputs

The same questions with their state padded by unrelated documents (served FP8):

Training rows were at most 32,768 tokens; the 61K and 92K levels are outside the training distribution. The untrained base was not run at these lengths. 240 JevBench-format questions of up to 96,161 tokens were all answered (0 failures, 0 refusals).

JevBench

JevBench (Benchmark Heaven, MIT) scores typed decisions by Capability = (Intelligence + Calibration) / 2; a Noul answer with P(yes) between 0.2 and 0.8 counts as an abstention.

• Official, JevBench v1.6.1. Benchmark Heaven ran these weights (revision 58d1545) itself on one H100 PCIe, on 1,500 items including the sealed ones, measured 2026-10-07 and published on the leaderboard:

  	Capability [95%]	Intelligence	Calibration	$/1k decisions
  Bobcat Flash 1.2, #6 open-weights	73.5 [69.5, 76.4]	65.8	81.2	0.048 (estimate)
  Quyet-1.0-Large, #1 open-weights	81.7	73.4	90.0	0.045 (estimate)
  Jev 1.13.0, reference, not ranked	77.1	63.6	90.6	0.032

  All 1,500 items were answered and none was invalid. Intelligence was 65.9 on the public items and 65.8 on the sealed ones (a gap of 0.11). 89.3% of Noul answers were decisive, and 91.9% of those were correct. Calibration is the weakest axis: Choice 77.0 (hard ECE 0.115), Score 81.7 (top ECE 0.149), Noul 84.9 (ECE 0.076, 10.7% abstentions). Raw latency per item on their GPU was p50 61 ms, p95 79 ms. The cost column is the board's estimate from the base model's list price (Bobcat produces no output tokens), not a measured bill. Ranks move as rows are added; #6 is the open-weights ranking on 2026-10-08.

• Public items (231), our reconstruction, not the official score. Three fresh servers as in the Quickstart (FP8, per-type temperatures, --max-model-len 98368) through the official harness at commit bb05a33, run once, after the release manifest was frozen:

  	Capability	Intelligence	Calibration
  Bobcat Flash 1.2, per-type temperatures (three runs: 68.1 / 68.6 / 67.9)	68.2	57.5	78.9
  Bobcat Flash 1.2, product temperature 0.9232 for every type	68.3	54.3	82.3
  Same base (Gemma 4 26B-A4B-it), zero-shot	67.5	67.6	67.4

  On these items, training raised Calibration by 11.4 points and lowered Intelligence by 10.1 against the untrained base: the base commits to more yes/no answers and gets more of the hard ones right. Between the three fresh servers 1 and 4 of 231 top answers changed. The reconstruction has no sealed half, no public-versus-sealed gap penalty and no judge tier, so it is not comparable with the public leaderboard; it came out 5.3 points below the official run.

• Our own JevBench-format development set (3,030 questions written for Bobcat, sharing no item with JevBench): Capability 86.0 (Intelligence 79.4, Calibration 92.6) against 67.2 for the same base zero-shot; Noul abstentions 7.6%, and 93.4% of the decided Noul answers correct. The same 240-question long set: 53.4 against 27.2. This set was used to select the model, and the public items show its gain does not carry over in full: it overstates the difference to the base and to Quyet-1.0-Large.

Latency and throughput

One H100 80 GB, vLLM 0.30.0, FP8, engine (no HTTP): 512 tokens, 8 candidates, 1 question, p50 / p95 19.0 / 19.5 ms; independent 1K-token requests up to 63,080 tokens/s (512 concurrent). Over loopback HTTP, one JevBench-format question at a time (3,030 questions): p50 39.3 ms, p95 142.5 ms.

The H100 figures were measured at a 32,832-token limit; at 98,368 the RTX PRO 6000 kept 85,169 MiB of its 97,887 MiB in use.

Bobcat Flash 1.1 -> 1.2

The same measurements of both models, paired where marked.

Flash 1.1's TypeSafe and injection values are its published ones; on our evaluation server the TypeSafe agreement was 91.19% and injection 9.7%, the same. Flash 1.1's public-231 value uses its release temperature and the same scorer.

Quyet-1.0-Large (our measurement, unofficial)

Quyet-1.0-Large (chinhnc/Quyet-1.0-Large, revision e1ecbbe, Apache-2.0) was first on the JevBench leaderboard on 2026-10-05. We ran it with its own published runtime (quyet 1.0.0, default settings, one H100) on the same items and scored both models the same way. These are our measurements, not JevBench's; the leaderboard's official figures use sealed items we have never seen. On the official board Quyet-1.0-Large scores 81.7 and Bobcat Flash 1.2 73.5 (above).

Quyet keeps the first 6,000 tokens of a state and accepts at most 10 options; Bobcat reads up to 98,368 tokens and 255 options. Quyet-1.0-Large is a separate project; nothing of it was used to train or tune Bobcat.

Training and distillation

• Base: Gemma 4 26B-A4B-it at revision 4d7ae4984b7db7de8f8457170b3f1a419ee76d52: 30 layers of sliding-window (1,024) and global attention, 128 experts with 8 active plus a dense MLP, about 4B active parameters.
• These weights are a merge of two rank-64 LoRA adapters on that base, each merged with bobcat.student_merge (float32 product and sum, BF16 storage), then combined tensor by tensor as 0.6 x W_A + 0.4 x W_B with scripts/arms12_interp.py (235 adapted tensors; the experts, embeddings and output head are the base's):
  • A (adapter_model.safetensors sha256 7f9ae4d8…a7f39e): trained for this release, below.
  • B (2304bd8e…9ca3): the adapter of sanghwa-na/bobcat-flash-1.1, unchanged.
  • Weights 0.5, 0.6, 0.7, 0.775 and 0.85 for A were measured; 0.6 is the only one that met every release criterion (development split, TypeSafe's examples, injection, long states). No JevBench item was used to choose it.
• Adapter A: LoRA rank 64, alpha 128, on the attention projections, the dense MLP and the expert router of all 30 layers (80.0M trainable parameters); one epoch, 1,922 steps of 32,768 tokens on each of eight GPUs, learning rate 2e-4, cosine decay.

  • Loss per question: KL(teacher || student) where a teacher distribution exists, plus 0.5 times the gold loss (cross-entropy for Choice and Noul, plus an ordinal ranked probability score for Score), plus 0.2 x KL(base || student) on every question to keep the base's behaviour where the data says nothing. Questions whose gold is computed by code use a one-hot target; probability questions use their exact distribution.

  • Augmentation: half of the Choice questions with their candidate order permuted and half of the object states with their field order permuted, replacing the originals.

  • Data (625,625 questions, rows of at most 32,768 tokens):

    Source	Questions
    The Flash 1.1 training corpus (public decisions, generated workflows, policy transfer, product tasks), filtered against every evaluation split and sealed final	447,714
    Generated decisions with gold computed by code (dates and numbers, extraction, policy and long policy, multi-hop tables, ordinal rubrics, exact probabilities, state tracking; 20 languages, 77% English), with "none of these" options; plus CLINC150 and MASSIVE intents	143,169
    The same generators at 8K to 32K tokens	1,151
    Long padded copies of corpus questions (2K to 30K tokens)	1,415
    Judgement questions (answer quality, ambiguity, trade-offs, traps, adversarial instructions, hard routing) from OpenAssistant oasst2, HH-RLHF, GSM8K and TruthfulQA, and generated ones	32,176

  • Labels and teachers: labels computed by code; teacher distributions from GLM-5.3-Flash (MIT, read at the first answer position without generating) for the judgement questions, and from the Bobcat 1.1 adapter (sha256 7351d959…83b2, temperature 1.2008) for the corpus questions. Where a human or code gold exists, the target is 0.5 x gold + 0.5 x teacher when the teacher's top answer equals the gold, and the gold alone when it does not; teacher-only questions are used only for hard routing.

• Calibration: per question type, Choice 0.5946, Noul 0.3536 and Score 0.5297, chosen on a held-out calibration split of JevBench-format decisions (8,906 questions, generated and public) for the highest JevBench-style Capability; and one product temperature, 0.9232, fitted by log-loss on the held-out calibration split of the six product tasks.
• Provenance: No output of Jev was used for training, distillation, reward or calibration, and no JevBench item was used for training, calibration, selection or prompt design; generated rows were audited against JevBench's public items (0 exact or 8-gram overlaps). No TypeSafe, SemIf or evaluation row was used. Tool-call review was never trained.

Limitations and risks

• Release criterion changed after measurement. TypeSafe agreement was to be at least 91.5%. After seeing that every 26B-A4B variant, including the untrained base, scored 89.1-90.3% on our setup, the criterion for this tier was set to the same-setup value of the earlier Flash model minus 1.0 point (90.19%); Flash 1.2 scores 90.27%. Under the original criterion it would not have been released.
• Classification (76.4% on development) and search passage selection remain the weakest product tasks.
• Long states cost 2.3 points at 30K and 4.3 points at 61K-92K tokens; nothing above 32,768 tokens was trained.
• Near-ties move between server runs. Three fresh FP8 servers changed 1 and 4 of 231 top answers on JevBench's public items, and the 240-question long set scored 49.7 on an RTX PRO 6000 against 53.4 on the evaluation server. Where an answer must be reproducible, treat answers with a top probability below about 0.8 as unsure.
• On the official JevBench board Flash 1.2 is 8.2 points below Quyet-1.0-Large (73.5 against 81.7), in Calibration (81.2 against 90.0) and in Intelligence (65.8 against 73.4). In our reconstruction on JevBench's public items it also scores below Bobcat 1.1 (68.2 against 73.8 on the same scorer), and its Intelligence there is lower than the untrained base's; our own development set, which favours it by 7.7 points over Quyet, does not predict this.
• The development split and the sealed final are Korean; English results come from TypeSafe's examples and our generated sets, which no person has reviewed.
• A typed answer can be the wrong valid option, and confidence is a concentration statistic, not the probability of being right. Compute arithmetic, counting and date comparison in code where it matters.

Intended use: small, repeated judgments whose admissible answers the application owns, where time per request matters - routing, guardrail screening, citation checks, tool-call review, classification - with the policy that acts on the answer kept in the application's code. Out of scope: generating text, open-ended question answering, and sole reliance on Bobcat Flash for safety-critical or legal decisions.

License and attribution

These weights are released under Apache-2.0 as a derivative of Gemma 4 26B-A4B-it by Google DeepMind, released under the Apache License 2.0; the license text is included as LICENSE, and NOTICE lists what was changed. Teachers: Bobcat adapters of Qwen3.8-27B (Apache-2.0) and GLM-5.3-Flash (MIT). Training data keep their own licenses: KLUE, KorNLI, ARC and SNLI (CC BY-SA 4.0), BoolQ (CC BY-SA 3.0), NSMC (CC0 1.0), MASSIVE, Banking77 and HelpSteer 2 and 3 (CC BY 4.0), CLINC150 (CC BY 3.0), MultiNLI (mostly the Open American National Corpus license, with fiction under CC BY-SA 3.0 and CC BY 3.0), OpenAssistant oasst2 and TruthfulQA (Apache-2.0), HH-RLHF and GSM8K (MIT). Attributions are in the code repository's THIRD_PARTY.md.

Bobcat is an independent project. It is not affiliated with or endorsed by TypeSafe AI, Benchmark Heaven, Google, the Qwen team, Z.ai, Anthropic, OpenAI or any other company named here; product names are their owners' trademarks and are used only to identify the models compared. TypeSafe's, SemIf's and JevBench's evaluation data are not redistributed here. The weights are provided as is, without warranty.

Citation

@misc{bobcatflash12,
  title        = {Bobcat Flash 1.2},
  author       = {{The Bobcat Authors}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sanghwa-na/bobcat-flash-1.2}}
}