BTL-4
A 35B agentic reasoning model from Bad Theory Labs, fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus.
Built for tool use, software engineering and long-horizon agent work.
Benchmarks
| Benchmark | BTL-4 | Base Ornith-1.0-35B | Harness |
|---|---|---|---|
| BFCL v4 (AST) | 73.5% | 69.2% | official ast_checker, all 1240 cases |
| LiveCodeBench v6 | 66.1% | — | official, 442 problems, 2024-08 → 2025-05 |
| SWE-bench Verified | 78.4% | — | official harness |
BFCL and LiveCodeBench were run in-house with the official scorers, full splits, no subsetting. The BFCL number is a paired comparison: identical harness, identical decoding, only the weights differ.
LiveCodeBench by difficulty
| pass@1 | |
|---|---|
| easy | 99.1% |
| medium | 86.7% |
| hard | 60.5% |
The set is 45% hard problems, which is what pulls the aggregate down.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "badtheorylabs/BTL-4"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16",
device_map="auto")
messages = [{"role": "user", "content": "Refactor this function to be pure."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=2048)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
Serving
vllm serve badtheorylabs/BTL-4 \
--max-model-len 131072 \
--enable-auto-tool-choice --tool-call-parser qwen3_xml \
--reasoning-parser qwen3 \
--trust-remote-code
llama.cpp, using the GGUF build:
llama-server -m BTL-4-IQ2_XXS.gguf --port 8080 \
--jinja \
--reasoning-format deepseek \
-c 32768 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0
Reasoning must be separated from content, on every stack. The chat template
strips reasoning from older turns, but only when the harness puts it in
reasoning_content. With vLLM that is --reasoning-parser qwen3; with
llama.cpp it is --reasoning-format deepseek. Without it, reasoning accumulates
into content each turn and the model repeats turns instead of terminating.
Generation settings
Ornith's published settings, used for every number above:
| temperature | 1.0 |
| top_p | 0.95 |
| context | 262144 native |
Give it room to think. LiveCodeBench improved 60.9% → 66.1% purely by raising the output budget from 16K to 32K. At 16K, 23.5% of problems were truncated mid-solution and scored zero. Hard problems reason longer; cutting them off costs real points.
What it is good at
- Tool calling — 73.5% BFCL v4 AST, +4.3 points over base
- Competitive programming — 99.1% easy / 86.7% medium on LiveCodeBench v6
- Long context — 262K native, and it uses it
What it is not
- Not a chat model. It reasons before answering and is verbose by default.
- Reasoning accumulates across agent turns. The chat template strips prior
reasoning from older turns, but this only works if your harness separates it
into
reasoning_content. With vLLM, that means--reasoning-parser qwen3. Without it, thinking lands incontent, accumulates every turn, and long agent runs degrade. - Token-hungry on hard problems. Budget accordingly.
Training
Fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus: candidate trajectories were kept only where the resulting code actually ran and passed its tests, so the reasoning that survived is reasoning that led somewhere.
Citation
@misc{btl4-2026,
title = {BTL-4: An Execution-Gated Agentic Reasoning Model},
author = {Bad Theory Labs},
year = {2026},
url = {https://huggingface.co/badtheorylabs/BTL-4}
}