Certo-R1 (Qwen3-4B)
Concise, verified reasoning for short calculation/compositional decisions. The escalation model of the Certo decision cascade: a lightweight decider handles the confident majority; low-confidence cases escalate here for a brief reasoned answer.
Credit / base model
Built on Qwen/Qwen3-4B (Alibaba Qwen team), used under
Apache-2.0. Certo-R1 is a LoRA fine-tune (rank 16) merged into the base weights. All credit for the base
model goes to the Qwen team; this repo adds only the concise-reasoning adaptation. Base revision 1cfa9a72.
What it does
Given a decision (context + options A/B/C…), it reasons in one or two short sentences and ends with
FINAL ANSWER: <letter>. On a held-out set of ten short calculation/compositional families it is
non-inferior to full verbose reasoning within a pre-registered −0.03 margin (cascade Δ +0.004,
95% CI [−0.004, +0.012]) while generating ~15× fewer tokens (≈32 vs ≈489, p50) and cutting routed
latency ~8× (mean) / ~11× (p95).
JevBench 1.4.2 (public suite, 231 tasks)
Measured locally on the public tiers (the official score additionally blends 308 sealed decisions held by Benchmark Heaven — not included here). Certo-R1 is a reasoner, not a calibrated decider: it emits an argmax answer, so its Calibration axis is ~0 and the geometric-mean composite is gated — read the per-tier Intelligence instead.
| tier | accuracy | Intelligence (chance-corrected) |
|---|---|---|
| easy | 0.979 | 97.1 |
| standard | 0.903 | 85.9 |
| hard | 0.595 | 38.9 |
| overall | 0.771 | — |
Speed axis 79.5 (p50 0.50 s), Cost axis 58.1 ($0.025 / 1000 decisions at a $0.04/M input tariff). The generative Certo-R1 reasons far above the shipped encoder Certo (JevBench Intelligence ~28), trading calibration for reasoning.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("altslate/certo-r1-qwen3-4b")
m = AutoModelForCausalLM.from_pretrained("altslate/certo-r1-qwen3-4b", torch_dtype="bfloat16", device_map="auto")
body = ("Is the minimum rest period satisfied?\n\nContext: A shift ends at 20:00 UTC. The next shift begins "
"at 06:00 CET (UTC+1) the following day. At least 11 hours of real rest are required.\n\nOptions:\n"
"A) at least 11 hours of real rest\nB) less than 11 hours of real rest\n\n"
"Reason in at most two short sentences, then end with a line 'FINAL ANSWER: <letter>'.")
ids = tok.apply_chat_template([{"role":"user","content":body}], return_tensors="pt",
add_generation_prompt=True, enable_thinking=False).to(m.device)
print(tok.decode(m.generate(ids, max_new_tokens=96, do_sample=False)[0][ids.shape[1]:], skip_special_tokens=True))
Decoding: greedy (temperature 0); the answer is the letter after FINAL ANSWER:.
Related: Certo-SR (experimental self-router)
A follow-up merges the decider into this backbone — one model with a fast option-scoring head plus this LM reasoning head, generating only when the head is unsure (~⅓ the serving memory). On a pre-registered fresh confirmation the self-route was not accuracy-non-inferior to always-reasoning (routing trades ~2.5 pp for speed/memory), so it is kept experimental. Certo-R1 remains the released reference.
Scope & caveats
- Validated on ten short calculation/compositional families; not a general chat model, not validated on long documents or other task types.
- Trained on deterministic oracle-verified rationales; the evidence is held-out generalization, not train loss.
- Emits an argmax answer + rationale, not a calibrated probability.
See certo_manifest.json for full training/eval provenance and hashes.