Sabi-Noema
Sabi-Noema is an agentic model built on top of Qwen3.8-27B using exclusively open-source RL environments.
With Sabi-Noema, we simultaneously push the frontier of small model capability and reasoning efficiency.
⚡ Performance at a Glance
- Terminal-Bench 2.1: 76.0% (vs. 73.0% base)
- DeepSWE: 47.5% (vs. 38.5% base)
- τ³-bench: 93.2% (vs. 79.1% base) | 43.5% on banking (vs. 39.8% base)
- Hallucination Rate: 15.1% on AA-Omniscience (vs. 30.4% base)
- Token Efficiency: 2.3× fewer output tokens than Qwen3.8-27B and 2.0× fewer than Swift 1.5 on the Terminal-Bench tasks both solve
Resources
- Sabi-Noema-MLA — the same model with multi-head latent attention, for many long sessions per GPU
Benchmarks
Sabi-Noema against Qwen3.8-27B and comparison models. Same harnesses for every model: Terminal-Bench 2.1 following Artificial Analysis's settings on H100 TP2 deployments with up to 8 tasks per server (Muse Glimmer on TP1); τ³-bench (airline, retail, telecom) with a GPT-4.1 user simulator; DeepSWE uncapped (ThinkingCap: 3-hour cap per task; seeds with an infrastructure failure scored as a model fail are excluded); ClawEval graded by the harness's LLM judge. GPT-6 Luna (openai/gpt-6-luna) was called through OpenRouter at high reasoning effort; Muse Glimmer 30B was served locally in BF16 at its highest reasoning setting with its 131K-token context, and tasks that overflowed it count as failures. Qwen reports 73.0 on Terminal-Bench 2.1 with its own Terminus setup and 42.2 on DeepSWE 1.1 with the Claude Code harness. Terminal-Bench 2.1 is averaged over 2–3 runs per open model and DeepSWE over 2–3 seeds for Sabi-Noema, Sabi-Noema-MLA and Qwen3.8-27B; GPQA-Diamond, IFBench and AA-LCR over 3 runs where we ran them.
| Benchmark | Sabi-Noema | Qwen3.8-27B | ThinkingCap | Swift 1.5 | Muse Glimmer 30B | GPT-6 Luna, high |
|---|---|---|---|---|---|---|
| SWE & Terminal | ||||||
| Terminal-Bench 2.1 (%) | 76.0 | 73.0 | 67.8 | 62.9 | 48.3 | 68.5 |
| DeepSWE (%) | 47.5 | 38.5 | 38.9 | 39.8 | 5.3 | 53.1 |
| Agentic | ||||||
| τ³-bench (/278) | 259 | 220 | 204 | 195 | 219 | 183 |
| τ³-bench banking (%, 97 tasks × 5) | 43.5 | 39.8 | 36.9 | 40.4 | 15.9 | 28.7 |
| ClawEval (/199) | 162 | 151 | 151 | 160 | 153 | 160 |
| Tool calling | ||||||
| BFCL v4 multi-turn | 63.9 | 59.6 | 58.8 | 62.6 | 55.8 | 44.9 |
| MCPMark (/55) | 48 | 36 | 42 | 37 | 32 | 38 |
| Reliability | ||||||
| AA-Omniscience index | +2.3 | −5.2 | −10.5 | −22.8 | −28.0 | +5.2 |
| Hallucination rate (%, lower is better) | 15.1 | 30.4 | 36.1 | 52.6 | 84.6 | 85.3 |
| Retained Capabilities | ||||||
| LiveCodeBench v6 | 88.0 | 89.7 | 88.9 | 89.1 | 64.0 | 88.0 |
| GPQA-Diamond | 90.6 | 89.2 | 88.0 | 88.6 | 83.5 | 89.4 |
| MMLU-Pro | 85.1 | 85.4 | 85.0 | 85.7 | 84.2 | 83.4 |
| AA-LCR | 82.3 | 83.0 | 82.0 | 82.7 | 81.3 | 83.0 |
| IFBench | 80.9 | 79.5 | 79.7 | 81.1 | 77.0 | 61.0 |
| Efficiency (lower is better) | ||||||
| TB2.1 output tokens per task, median | 29.3K | 65.3K | 51.0K | 57.1K | 20.6K | — |
| τ³-bench tokens per conversation, mean | 3.9K | 4.2K | 3.0K | 3.6K | 2.7K | — |
| DeepSWE output tokens per task, mean | 82.7K | 118.8K | 103.7K | — | — | — |
Efficiency
Sabi-Noema writes much less to get there. On the tasks both models solve, it uses less than half the output tokens of Qwen3.8-27B and about half those of Swift 1.5 on Terminal-Bench 2.1, and 29% fewer than Qwen3.8-27B on DeepSWE:
The Terminal-Bench token row in the benchmark table is a median over all tasks, which includes failed attempts that run long; comparing only tasks both models solve measures the same work.
Model Overview
| Item | Specification |
|---|---|
| Base model | Qwen/Qwen3.8-27B |
| Parameters | ~27B, dense hybrid |
| Context length | 262K tokens |
| Precision | BF16 weights (~55 GB) |
| Release | October 2026 |
| Serving | vLLM 0.31.0 (stock), Transformers, llama.cpp (GGUF conversion) |
| License | Apache 2.0 |
Quickstart
Use the same generation parameters as Qwen3.8-27B; they are the defaults in generation_config.json.
A non-thinking mode (chat_template_kwargs={"enable_thinking": false}) is supported by the chat template but was not evaluated.
vLLM (one 80 GB GPU holds the ~55 GB of BF16 weights plus long contexts):
pip install "vllm==0.31.0"
vllm serve sabicap/sabi-noema --max-model-len 262144 --max-num-seqs 32 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
--max-num-seqs 32: vLLM's default of 1024 exceeds the linear-attention state blocks available on one GPU. The evaluations above also used --default-chat-template-kwargs '{"preserve_thinking": false}' (earlier turns' reasoning is not replayed); examples/serve_vllm.sh runs the command with it.
Chat with any OpenAI client; tool calls use the Qwen3.8 XML format:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
tools = [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city.",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
resp = client.chat.completions.create(
model="sabicap/sabi-noema", messages=[{"role": "user", "content": "What's the weather in Paris?"}], tools=tools,
temperature=1.0, top_p=0.95, max_tokens=32768, reasoning_effort="xhigh", extra_body={"top_k": 20})
msg = resp.choices[0].message
print(msg.reasoning, msg.tool_calls or msg.content)
examples/ has the full serve script, a thinking/multi-turn client and a complete tool round trip.
Transformers: AutoModelForImageTextToText.from_pretrained("sabicap/sabi-noema", dtype="bfloat16", device_map="auto") loads the full checkpoint; AutoModelForCausalLM loads the language model alone. See examples/transformers_generate.py.
llama.cpp: convert with convert_hf_to_gguf.py --no-mtp (the checkpoint has no MTP head) and serve with llama-server --jinja; see examples/llama_cpp.md.
Use Cases
- Long jobs in a shell or with tools, where a task takes many actions and has to be checked at the end
- Customer-facing agents bound by policy, including difficult customers, with correct tool calls over many turns
- Assistants that should say when they do not know: half Qwen3.8-27B's hallucination rate
- Cost-sensitive agentic serving: fewer output tokens per task than Qwen3.8-27B
- Many long sessions on one GPU: Sabi-Noema-MLA shrinks the attention cache from 64 KiB per token across the 16 full-attention layers to 18 KiB (3.6× smaller), or 9 KiB with an FP8 cache (7.1×), so many more agent sessions fit on the same card