Bobcat 1.3

Bobcat 1.3 · Bobcat 1.3 FP8 · Bobcat 1.2 · Bobcat Flash 1.2 · Code on GitHub

Bobcat is a typed-decision model. You send a state (text or JSON) and questions whose possible answers you define in the request; Bobcat returns a probability for every answer you named, and nothing else. It never generates text: it reads the logits of the offered candidates at the first answer position of one forward pass, and the host builds a closed JSON reply from your own names. An answer can be wrong, but it cannot be malformed.

This repository, sanghwa-na/bobcat-1.3, holds Bobcat 1.3 as ready-to-serve BF16 weights on google/gemma-4-31B-it (Apache-2.0) at revision 842da3794eaa0b77d5f08bae87a17459d91ff475. It keeps Bobcat 1.2's product behaviour and adds broader decision skills (tool use, claims against evidence, contracts, science and commonsense questions, stance, humor) learned from licensed, human-written public data. vLLM serves it in FP8, and it reads inputs of up to 98,368 tokens without truncating them.

Highlights

Jev figures are never used; Jev was never called. Differences between Bobcat 1.3 and 1.2 on these sets are small and mostly inside their 95% intervals (see Evaluation).

What it does

The request and reply use the shapes of TypeSafe's published System One HTTP API, so the official typesafe-sdk works against a Bobcat server by changing its base URL. Inputs over the server's limits are refused with HTTP 422 and never truncated.

Quickstart: serve Bobcat 1.3 on one GPU

curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH="$HOME/.local/bin:$PATH"   # uv
git clone https://github.com/foxl-ai/bobcat && cd bobcat
export UV_PYTHON_PREFERENCE=only-managed
uv venv --python 3.12 .venv-serve
uv pip install --no-config --python .venv-serve/bin/python vllm==0.30.0 fastapi uvicorn scipy jinja2 \
  "tokenizers>=0.21" huggingface_hub typesafe-sdk==0.7.1
export PYTHONPATH=$PWD/src PY=.venv-serve/bin/python

# These weights (about 62 GB), then the typed-decision server, FP8 at load, 96K states
$PY -c "from huggingface_hub import snapshot_download as s; s('sanghwa-na/bobcat-1.3', local_dir='bobcat-1.3')"
VLLM_USE_FLASHINFER_SAMPLER=0 $PY -m bobcat.flash_server --engine vllm --model bobcat-1.3 \
  --compiler-model bobcat-1.3/compiler --quantization fp8 \
  --temperature choice=1.007793,noul=0.445449,score=1.023435 \
  --name bobcat-1.3 --release-date 2026-10-11 --max-num-seqs 64 --max-model-len 98368 \
  --schedule all --engine-arg max_num_batched_tokens=16384 --engine-arg attention_backend=TRITON_ATTN \
  --host 127.0.0.1 --port 8000 --local

Then call it with the official SDK as on the Bobcat Flash 1.2 card, with TYPESAFE_DEFAULT_MODEL="bobcat-1.3".

• --temperature choice=1.007793,noul=0.445449,score=1.023435 applies one temperature per question type: Choice and Score fitted by log-loss on the even half of the public LocalLLaMA/typed-decisions test set (never trained on), Noul on a held-out calibration split of JevBench-format decisions. A temperature never changes an argmax. --temperature 1.2227 applies the single temperature fitted on the held-out calibration split of the product tasks.
• --quantization fp8 is vLLM's online FP8 of these BF16 weights (about 31 GB on the GPU). A pre-quantized FP8 checkpoint is at sanghwa-na/bobcat-1.3-fp8. On an 80 GB GPU add --engine-arg kv_cache_dtype=fp8.
• --max-model-len 98368 reads states up to about 96K tokens in full; longer inputs are refused with HTTP 422. --local disables the shared secret; use it only on a loopback interface.

Running on AWS

One RTX PRO 6000 Blackwell 96 GB (g7e.4xlarge) or one H100 80 GB (p5.4xlarge, FP8 KV cache) runs the Quickstart; the evaluation servers ran one per GPU on B200 hosts (p6-b200.48xlarge). Use a Deep Learning AMI with a recent NVIDIA driver and about 140 GB of disk.

How it was made

• Three LoRA arms, averaged. Each arm is a rank-16 LoRA (alpha 32) on the base, trained for one epoch (lr 1e-4, 32,768 tokens per GPU on eight GPUs) with the Bobcat 1.2 recipe: KL to a Bobcat teacher where one exists, the gold loss, Score RPS, and KL to the frozen base on every question except the injection pairs. The arms differ only in how many new skill questions they saw (27,000 or 54,000) and the base-KL weight (1.0 or 0.5). Bobcat 1.3 is their uniform average in weight space (each arm merged with bobcat.student_merge, then a float32 weighted sum, BF16 storage). The average was chosen over each single arm by a rule written before it was measured: the development Decision Index half plus six public decision sets that no Bobcat model ever trained on.
• Data. Bobcat 1.2's training rows without the rows from ARC and BANKING77 (benchmarks of the Decision Index) or overlapping its items (54,732 questions), plus up to 54,000 new skill questions from 17 public sources with commercial-use licences and human-written text and labels: MASSIVE (tool choice, call review and when to call, built from human intents and slots), CommonsenseQA, GAP, QASC, MedMCQA, MedQA, ScienceQA, AQuA-RAT, WiCE, VitaminC, FaithDial, IBM argument stance, a sarcasm headline set, MAUD, LEDGAR, UNFAIR-ToS and ColBERT humor. Datasets whose text was written by a model were left out. Every new question was checked against the Decision Index 0.3 public suite (140,620 requests), the JevBench public items and our evaluation sets (exact strings and word 8-grams); 765 were removed.
• Labels: the source datasets' own labels only for the new questions; the Bobcat 1.2 rows keep their gold and the Bobcat 1.1 teacher distributions (our earlier model). No GLM, gpt-oss, closed-API or Jev output was used, and no JevBench or Decision Index item was used for training, calibration or prompts.

Evaluation

Differences are paired, with 95% bootstrap intervals over source components. The development and sealed-final figures use the evaluation path (these merged BF16 weights, bobcat.flash_train --eval-only); TypeSafe, injection, long-state, JevBench-format and Decision Index figures use the weights served in FP8 over HTTP.

Sealed final

A sealed split of the product evaluation (1,609 decisions, four tasks, Korean), opened once, after the release manifest that binds these weights, their temperatures and the analysis was frozen. Failures count as wrong (there were none).

• Paired: Bobcat 1.3 minus Bobcat 1.2 -0.27 [-0.75, +0.18]; minus Bobcat 1.1 -0.07 [-1.37, +1.10]; minus Flash 1.1 +1.36 [-0.05, +2.76]. Without the 50 pre-declared rows whose text also appears in a training file (none added by this release's data): 94.47%.
• The release bar fixed before the opening was 92.21%; it was met. Log-loss 0.335, ECE 0.015 at the product temperature.

Development and gates

Tool-call review was trained for this release (the MASSIVE tool questions and the product task's own rows are different data); in Bobcat 1.2 it was held out.

Decisions outside the product: held-out and public sets

• The Decision Index half was used, with the six held-out sets, to choose among the candidates by a rule written before they were measured, so the 59.18 is optimistic for this model. Areas against Bobcat 1.2 on that half: knowledge +0.1, language +0.7, retrieval -1.0, tools +1.5, arts -0.8.
• Held-out sets per source (Bobcat 1.3 / 1.2): GoEmotions 53.9 / 54.7, StrategyQA 59.3 / 53.3, QuaRTz 88.0 / 87.3, PAWS 63.3 / 62.7, ETHICS justice 77.3 / 75.3, SNIPS 97.8 / 97.8.

JevBench-format decisions

• Public items (231): our measurement, unofficial; not a JevBench rank. Three fresh servers as in the Quickstart through the official harness at commit bb05a33, run once after the manifest was frozen: Capability 77.2 (three runs 76.7 / 76.5 / 78.3; Intelligence 69.0, Calibration 85.3). Bobcat 1.2 measured the same way: 77.8. Paired: minus the untrained base +1.3 [-7.2, +8.5]; minus Quyet-1.0-Large (its own runtime, our measurement) +1.6 [-1.3, +7.9].
• Our own JevBench-format development sets (no JevBench item): Capability 78.7 (Bobcat 1.2 79.3), hardest tier 73.2 (73.7), judgement questions 76.4 (75.7), 61K-96K-token questions 57.7 (48.8).

Latency and throughput

These were measured on one B200; the Bobcat 1.2 card's H100 80 GB and RTX PRO 6000 figures are for the same architecture and serving command.

Limitations and risks

• The gains over Bobcat 1.2 are small; most single comparisons are inside their 95% intervals.
• The Decision Index figures above are our runs of the public kit on its public items; half of them were used to choose among candidates, so they are optimistic. The official index adds private sets only the maintainers can run.
• Tool-call review is now trained (it was held out in 1.2); the transfer check moved to the typed-decisions set.
• Wrong answers planted in a document under review still win often on the external-document screening task; long states cost accuracy beyond the trained 32,768 tokens.
• A typed answer can be the wrong valid option, and confidence is a concentration statistic, not the probability of being right.

License and attribution

These weights are released under Apache-2.0 as a derivative of Gemma 4 31B-it by Google DeepMind (Apache License 2.0); LICENSE and NOTICE are included. Training data keep their own licenses (see the code repository's THIRD_PARTY.md). Bobcat is an independent project, not affiliated with or endorsed by TypeSafe AI, Benchmark Heaven, the Decision Index maintainers, Google or any other company named here. The weights are provided as is, without warranty.

Citation

@misc{bobcat13,
  title        = {Bobcat 1.3},
  author       = {{The Bobcat Authors}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sanghwa-na/bobcat-1.3}}
}