Bobcat 1.2

Technical write-up: foxl.ai/blog/bobcat-typed-decisions

Bobcat 1.2 · Bobcat 1.2 FP8 · Bobcat Flash 1.2 · Try it in the Space · Code on GitHub

Bobcat is a typed-decision model. You send a state (text or JSON) and questions whose possible answers you define in the request; Bobcat returns a probability for every answer you named, and nothing else. It never generates text: it reads the logits of the offered candidates at the first answer position of one forward pass, and the host builds a closed JSON reply from your own names. An answer can be wrong, but it cannot be malformed.

This repository, sanghwa-na/bobcat-1.2, holds Bobcat 1.2 as ready-to-serve BF16 weights on google/gemma-4-31B-it (Apache-2.0) at revision 842da3794eaa0b77d5f08bae87a17459d91ff475, a dense 31B model. vLLM serves it in FP8, and it reads inputs of up to 98,368 tokens without truncating them. The compiler, servers and evaluation code are at github.com/foxl-ai/bobcat.

Highlights

Jev figures are TypeSafe's own published answers; Jev was never called. The same-base baseline is untrained Gemma 4 31B-it with the same compiler and readout. See Limitations for what this model does not do well, including a release criterion that was changed after measurement.

What it does

The request and reply use the shapes of TypeSafe's published System One HTTP API, so the official typesafe-sdk works against a Bobcat server by changing its base URL. Inputs over the server's limits are refused with HTTP 422 and never truncated.

Quickstart: serve Bobcat 1.2 on one GPU

A clean install with the commands below was tested on one RTX PRO 6000 Blackwell 96 GB after these weights were published: a fresh Python 3.12 venv with exactly the install line below, a clone of the public code repository, the download line, then the server command. The SDK example answered payments, and TypeSafe's 20 workflow cases agreed with the reference on 90.27% of 329 questions over loopback HTTP. The same server command gave every served figure on this card (one GPU per server, vLLM 0.30.0).

curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"   # uv
git clone https://github.com/foxl-ai/bobcat && cd bobcat
export UV_PYTHON_PREFERENCE=only-managed   # a uv-managed Python ships the headers Triton compiles against
uv venv --python 3.12 .venv-serve
uv pip install --no-config --python .venv-serve/bin/python vllm==0.30.0 fastapi uvicorn scipy jinja2 \
  "tokenizers>=0.21" huggingface_hub typesafe-sdk==0.7.1
export PYTHONPATH=$PWD/src PY=.venv-serve/bin/python

# These weights (about 62 GB), then the typed-decision server, FP8 at load, 96K states
$PY -c "from huggingface_hub import snapshot_download as s; s('sanghwa-na/bobcat-1.2', local_dir='bobcat-1.2')"
VLLM_USE_FLASHINFER_SAMPLER=0 $PY -m bobcat.flash_server --engine vllm --model bobcat-1.2 \
  --compiler-model bobcat-1.2/compiler --quantization fp8 \
  --temperature choice=1.071833,noul=0.264866,score=1.157635 \
  --name bobcat-1.2 --release-date 2026-10-09 --max-num-seqs 64 --max-model-len 98368 \
  --schedule all --engine-arg max_num_batched_tokens=16384 --engine-arg attention_backend=TRITON_ATTN \
  --host 127.0.0.1 --port 8000 --local

Then call it with the official SDK exactly as on the Bobcat Flash 1.2 card, with TYPESAFE_DEFAULT_MODEL="bobcat-1.2".

Notes:

• bobcat.flash_server serves every Gemma 4 Bobcat model; the name is historical.
• --temperature choice=1.071833,noul=0.264866,score=1.157635 applies one temperature per question type: Choice and Score fitted by log-loss on the test half of the public LocalLLaMA/typed-decisions set (never trained on), Noul on a held-out calibration split of JevBench-format decisions for the best JevBench-style Capability. A temperature never changes an argmax. --temperature 1.0833 instead applies the single temperature fitted by log-loss on the held-out calibration split of the six product tasks.
• --quantization fp8 is vLLM's online dynamic FP8 of these BF16 weights (about 31 GB on the GPU), the served configuration. A pre-quantized FP8 checkpoint is at sanghwa-na/bobcat-1.2-fp8 (33.3 GB to download; development accuracy -0.22 points [-0.52, +0.06] against these weights, see its card). On an 80 GB GPU add --engine-arg kv_cache_dtype=fp8: one H100 80 GB then holds 212,884 tokens of KV cache, 2.2 requests of 98,368 tokens.
• --max-model-len 98368 reads states up to about 96K tokens in full (state plus question); longer inputs are refused with HTTP 422 and never truncated. On a GPU with less memory, lower it; the limit only moves the refusal point.
• --local disables the shared secret the servers otherwise require; use it only on a loopback or private interface. Run the server from the repository root, or pass --identifiers bobcat-1.2/bobcat-identifiers.json.

Running on AWS

These are ordinary GPU Linux hosts; the Quickstart above is the whole recipe.

• Amazon EC2. One H100 80 GB (p5.4xlarge) runs the server at 98,368 tokens with an FP8 KV cache and gave the latency figures above; one RTX PRO 6000 Blackwell 96 GB (g7e.4xlarge) runs it with the default KV cache; the evaluation servers ran one per GPU on B200 hosts (p6-b200.48xlarge). Use a Deep Learning AMI with a recent NVIDIA driver and allow about 140 GB of disk.
• Amazon SageMaker AI. The same commands run in a JupyterLab space or notebook instance of an equivalent GPU type. A real-time endpoint needs a custom container that runs bobcat.flash_server behind SageMaker's /invocations and /ping routes; we have not published or tested one.

Evaluation

Differences are paired, with 95% bootstrap intervals over source components. The development and sealed-final figures use the evaluation path (these merged BF16 weights, bobcat.flash_train --eval-only); TypeSafe, injection, long-state and JevBench-format figures use the weights served in FP8 over HTTP.

Sealed final

A sealed split of the product evaluation (1,609 decisions, four tasks, Korean), opened once, after the release manifest that binds these weights, their temperatures and the analysis was frozen. Nothing was changed after it. Failures count as wrong (there were none).

• Paired, over source components: Bobcat 1.2 minus Bobcat 1.1 +0.20 [-1.15, +1.41]; minus Bobcat Flash 1.1 +1.63 [+0.19, +3.07].
• Pre-declared sensitivity analysis, without the 50 rows whose text also appears in a training file (all in the earlier training corpora; none added by this release's data): 94.74%, +0.18 [-1.28, +1.40] over Bobcat 1.1 and +1.63 [+0.17, +3.04] over Flash 1.1.
• The release bar fixed before the opening was 92.21%; it was met.
• Served in FP8 (vLLM, engine path), the same decisions score 94.70%.
• Log-loss 0.318 and ECE 0.015 at the product temperature (Bobcat 1.1 0.307 / 0.008, Flash 1.1 0.397 / 0.009).

Six-task development evaluation

The product evaluation's development split (3,188 decisions, Korean), the same split the release gates used.

Same base zero-shot: 90.44%. Bobcat Flash 1.1: 92.07% (Bobcat 1.2 +0.88 [-0.15, +2.05]). Bobcat 1.1: 93.69% (its own release evaluation, not paired here).

TypeSafe's published workflow examples

20 cases, 329 questions answered by every published system (one question = 0.30 points). Agreement with TypeSafe's reference answers: 90.88% (299 of 329), from the release gate run (FP8 at load, the product temperature, vLLM 0.30.0, one B200). On the same server setup, Bobcat 1.1 as released (sanghwa-na/bobcat-1.1-fp8, its own server and temperature) agreed on 91.49% and the untrained base on 91.49%; Jev's published answers agree on 90.9%.

Wrong answers named inside the state

300 questions, each with three attacks that name a wrong answer inside the state (an administrator verdict, a sentence inside the text, a note to an AI grader). Attack success: 6.4% (same base zero-shot 46.4%; Bobcat 1.1 7.8%). Accuracy: clean 92.7%, administrator verdict 90.3%, inside the text 84.3%, note to an AI grader 85.3%.

• Almost all remaining successes are on the external-document screening task, where the state is the document under review: 37.6% of its 141 attacked questions. On the other five tasks the attack succeeds on 0.3% (2 of 720).

Long inputs

The same questions with their state padded by unrelated documents (served FP8):

Training rows were at most 32,768 tokens; the 61K and 92K levels are outside the training distribution. 240 JevBench-format questions of up to 96,161 tokens were all answered (0 failures, 0 refusals). On one H100 80 GB (FP8 KV cache) the eight longest of them (92,219 to 94,084 tokens) were all answered, in 79 to 82 seconds each.

JevBench-format decisions

JevBench (Benchmark Heaven, MIT) scores typed decisions by Capability = (Intelligence + Calibration) / 2; a Noul answer with P(yes) between 0.2 and 0.8 counts as an abstention.

• Public items (231): our measurement, unofficial; not a JevBench rank. Three fresh servers as in the Quickstart (FP8, per-type temperatures, --max-model-len 98368) through the official harness at commit bb05a33, run once, after the release manifest was frozen:

  	Capability	Intelligence	Calibration
  Bobcat 1.2, per-type temperatures (three runs: 78.4 / 77.5 / 77.5)	77.8	68.3	87.3
  Bobcat 1.2, product temperature 1.0833 for every type	75.6	62.2	89.0
  Same base (Gemma 4 31B-it), zero-shot	75.8	72.7	79.0
  Bobcat 1.1, as released	73.8	63.6	84.0

  • Paired over items: Bobcat 1.2 minus the same base +1.9 [-6.1, +8.5]; minus Bobcat 1.1 +3.9 [-5.0, +13.3]; minus Quyet-1.0-Large +2.3 [-1.4, +8.9].
  • On these items, training raised Calibration by 8.3 points and lowered Intelligence by 4.5 against the untrained base. No Noul answer was left between 0.2 and 0.8. Between the three fresh servers 2 of 231 top answers changed (the second and third runs were identical).
  • The reconstruction has no sealed half, no public-versus-sealed gap penalty and no judge tier, so it is not comparable with the public leaderboard.

• Our own JevBench-format development sets (written for Bobcat, sharing no item with JevBench; used in the release gates, so they favour the trained model):

  • 3,030 questions: Capability 79.3 (Intelligence 70.9, Calibration 87.7) against 73.8 for the same base zero-shot (+5.5 [+3.2, +7.3]); Noul abstentions 6.1%, and 87.3% of the decided Noul answers correct.
  • Its hardest tier: 73.7 against 66.6.
  • 1,189 judgement questions (answer quality, ambiguity, trade-offs, traps): 75.7 against 71.4.
  • 240 questions of 61K to 96K tokens: 48.8 against 39.5.

• No official JevBench run of this model exists.

Latency and throughput

Bobcat 1.1 -> 1.2

Bobcat 1.1's long-state figures were measured on another padded set and are not repeated here.

Quyet-1.0-Large (our measurement, unofficial)

Quyet-1.0-Large (chinhnc/Quyet-1.0-Large, revision e1ecbbe, Apache-2.0) was first on the JevBench leaderboard on 2026-10-05. We ran it with its own published runtime (quyet 1.0.0, default settings, one H100) on the same items and scored both models the same way. These are our measurements, not JevBench's; the leaderboard's official figures use sealed items we have never seen.

Quyet keeps the first 6,000 tokens of a state and accepts at most 10 options; Bobcat reads up to 98,368 tokens and 255 options. Quyet-1.0-Large is a separate project; nothing of it was used to train or tune Bobcat.

Training and distillation

• Base: Gemma 4 31B-it at revision 842da3794eaa0b77d5f08bae87a17459d91ff475, a dense model; its tokenizer, tokenizer configuration and chat template are byte-identical to Gemma 4 26B-A4B-it's, so both Bobcat 1.2 models share one compiler.
• Weights: one rank-16 LoRA adapter (adapter_model.safetensors sha256 c1cbdf4d…e2f8) merged into the base with bobcat.student_merge (float32 product and sum, BF16 storage; 410 adapted tensors).
• Adapter: LoRA rank 16, alpha 32 (122.4M trainable parameters); one epoch, 243 steps of 32,768 tokens on each of eight GPUs, learning rate 1e-4.

  • Loss per question: KL(teacher || student) where a teacher distribution exists, plus 0.5 times the gold loss (cross-entropy for Choice and Noul, plus an ordinal ranked probability score for Score), plus 1.0 x KL(base || student) on the corpus and product questions to keep the base's behaviour where the data says nothing.

  • Augmentation: half of the Choice questions with their candidate order permuted and half of the object states with their field order permuted, replacing the originals.

  • Data (62,464 questions, 57.6M tokens, rows of at most 32,768 tokens):

    Source	Questions
    Injection pairs: a wrong answer planted inside another task's text, the gold unchanged, and the clean question it was made from; gold only, no base KL, weighted 2x	15,770
    A task-balanced sample of the Flash 1.1 training corpus's public-data tasks (NLI, sentiment, intents, topic, QA, preference, generated workflows, policy transfer)	40,000
    Product tasks (search, citation, document screening, routing), at most 1,500 per task	6,000
    Long padded copies of product questions (2K to 30K tokens)	694

    Every row whose text meets a development, calibration or final split was removed first.

  • Labels and teachers (every training row traced to its source row and to the teacher files the trainer read):

    • The 15,770 injection pairs: gold only, the source dataset's label or a construction rule (KLUE MRC, Wizard of Seoul, publisher categories, public and policy-transfer questions); the planted name never changes it.
    • The other 46,694 questions: a teacher distribution from the Bobcat 1.1 adapter (sha256 7351d959…83b2, temperature 1.2008) for every one, plus a gold where one exists: 41,610 dataset labels or rule and generator golds, 3,402 generated score means. 1,682 generated questions have no gold, and the Bobcat 1.1 distribution is their only target.
    • Where a gold exists the target is KL to the teacher plus 0.5 x the gold loss.
    • No GLM-5.3-Flash, gpt-oss or other third-party model output was used; Bobcat 1.1, the only teacher, is our earlier model (see its card for its own training).
• Calibration: per question type, Choice 1.0718 and Score 1.1576, fitted by log-loss on the even half of the LocalLLaMA/typed-decisions test set (Apache-2.0, never trained on), and Noul 0.2649, chosen on a held-out calibration split of JevBench-format decisions for the highest JevBench-style Capability; and one product temperature, 1.0833, fitted by log-loss on the held-out calibration split of the six product tasks.
• Provenance: No output of Jev was used for training, distillation, reward or calibration, and no JevBench item was used for training, calibration, selection or prompt design. No TypeSafe, SemIf or evaluation row was used. Tool-call review was never trained.

Limitations and risks

• Release criterion changed after measurement. TypeSafe agreement was to be at least 91.5%. Both trained candidates fell short (91.19% and 90.88%; the untrained base scored 91.49%), and the criterion for this model was then set to Bobcat 1.1 as released, measured on the same server setup, minus 1.0 point: 91.49% - 1.0 = 90.49%. Bobcat 1.2 scores 90.88%. Under the original criterion it would not have been released. Of the two candidates that met the changed criterion, the one with the higher judgement-set Capability was chosen, as the plan said.
• Noul temperature chosen differently than planned. The plan fitted all three per-type temperatures on typed-decisions, Noul under a cap of 5% abstentions. No temperature met that cap on this set (its Noul gold is a soft label from several annotators; the lowest abstention was 6.7%), so Noul uses the calibration-split value. This was decided before the sealed final and the public items were opened.
• Wrong answers planted in a document under review still win 37.6% of the time on the external-document screening task, close to the untrained base; on the other tasks 0.3%.
• Classification (78.8% on development) and search passage selection remain the weakest product tasks; development macro is 0.75 points below Bobcat 1.1's.
• Long states cost 2.4 points at 30K and 5.7 to 6.3 points at 61K-92K tokens; nothing above 32,768 tokens was trained. A 92K-token request takes about 80 seconds on one H100 80 GB.
• Near-ties move between server runs. Fresh FP8 servers are not bit-identical: 2 of the 231 public JevBench answers changed between three servers, and on TypeSafe's 329 questions one answer is 0.3 points (fresh Flash 1.2 servers ranged over 89.97-90.6%).
• JevBench numbers here are our reconstruction, not the leaderboard's; the leaderboard uses sealed items we have never seen.
• The finals and the development split are Korean; English results come from TypeSafe's 20 cases and the JevBench-format sets. The development sets are generated or adapted, not human-reviewed.
• A typed answer can be the wrong valid option, and confidence is a concentration statistic, not the probability of being right.

Intended use and out of scope: as for Bobcat Flash 1.2.

License and attribution

These weights are released under Apache-2.0 as a derivative of Gemma 4 31B-it by Google DeepMind, released under the Apache License 2.0; the license text is included as LICENSE, and NOTICE lists what was changed. Teacher: the Bobcat 1.1 adapter of Qwen3.8-27B (Apache-2.0). Training data keep their own licenses, as listed on the Bobcat Flash 1.1 card and in the code repository's THIRD_PARTY.md.

Bobcat is an independent project. It is not affiliated with or endorsed by TypeSafe AI, Benchmark Heaven, Google, the Qwen team or any other company named here; product names are their owners' trademarks and are used only to identify the models compared. TypeSafe's and JevBench's evaluation data are not redistributed here. The weights are provided as is, without warranty.

Citation

@misc{bobcat12,
  title        = {Bobcat 1.2},
  author       = {{The Bobcat Authors}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sanghwa-na/bobcat-1.2}}
}