OsaurusAI/Raptor-0.6.1-preview-JANG_6M

Raptor 0.6.1 preview — a lightly tuned Spark-X2.5-4B, quantized to 3.41 GiB for Apple Silicon and framed for the Osaurus harness.

Raptor 0.6.1 preview is a supervised fine-tune of XHToken/Spark-X2.5-4B, a 4.112B dense reasoning and tool-use model. The tune is deliberately light: it touches attention only — the fused q_k_v_proj and out_proj of all 36 layers — and leaves the MLP, the tied embedding, every norm and the per-head attention gate byte-identical to the base. Tokenizer and chat template are unchanged from the base as well.

This is a JANG_6M bundle: attention and the tied embedding at 8-bit, the MLP carrying the 6-bit budget, every group scale in bfloat16, and the per-head gate and all norms kept in full precision.

This is a different backbone from Raptor 0.6-preview. Despite the adjacent version number, Raptor 0.6.1 preview runs on the spark2_5 architecture, where Raptor 0.6-preview used a looped-transformer backbone and Raptor v0.5 a hybrid one. It is a separate line, not a continuation of those weights.

Quality — measured, not estimated

15,839 teacher-forced positions on held-out prompts, scored against the model's own bf16 weights. Calibration and evaluation prompts come from disjoint corpora, asserted at eval time.

Decode is the median of 4 probes at a fixed condition (512-token prompt, 128 generated), first probe discarded, on an M5 Max; spread across the retained probes was 0.6%.

The margin-conditioned flip curve decreases monotonically across the first five bins (13.68% → 0.00%), then reads 0.04% in the final [8, ∞) bin. That final tick is one token out of 2,611 and is reported here rather than smoothed away; at that count it is sampling noise, not a structural defect.

There is no stock-MLX comparison row, and that is not an omission. mlx-lm has no spark2_5 architecture — not in any release, not on main — so no stock MLX quantization of this model exists to score against. The MLX runtime ships with the Osaurus harness.

Calibration

1,759,685 calibration tokens across code, agentic and tool-call transcripts, academic multiple-choice, general chat, Chinese, long-context, science and security prompts; 7.3% of tokens self-generated. One capture drives activation-aware scaling, per-channel importance and the error-corrected weight fit together, covering all 36 layers.

The base model's calibration was not reused. The tune moved attention, so the activation statistics moved with it; this bundle was captured against its own weights.

Applied: activation-aware scaling on 72/72 norm fold sites and 36/36 attention gate projections, error-corrected fitting on 180/181 tensors. The single holdout is the tied embedding — a lookup with no input activation to condition on, so it is fit directly.

Running it in Osaurus

Requires Osaurus 0.25.0 or newer — that is the first release whose engine carries the spark2_5 runtime. osaurus.json in this repo declares the requirement so the harness can tell you before it tries to load.

Point Osaurus at the repo and it serves with the sampling, reasoning and tool-call contract below already declared in the bundle: jang_config.json carries the parser names, dialect and delimiters, so the harness wires itself with no per-model configuration.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("OsaurusAI/Raptor-0.6.1-preview-JANG_6M")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is 84 * 3 / 2?"}],
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

A bare mlx_lm.load will not resolve spark2_5 on its own — the runtime ships with the harness.

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=-1 (no truncation, full-vocab top-p), no repetition penalty. Vendor defaults, carried verbatim in both generation_config.json and jang_config.json.
• Reasoning is ON by default. The template force-opens the rail: the generation prompt ends with <|Bot|><think>, and with enable_thinking=false it ends <|Bot|></think>. A reasoning parser that waits for a literal <think> in the output stream will never see one, because the opener is in the prompt. Reasoning off is a supported, tested lane.
• Give reasoning room. Reasoning traces on hard prompts can exceed 8,192 output tokens. Budget for real input plus output inside the context window rather than silently truncating history.
• Tools render into the first system block (## Tools plus a <tools> JSON list). Adding or removing a tool rewrites the system prefix, so a tool-set change is a full re-prefill, not a suffix append — any prefix-cache reuse accounting must treat it that way.
• Tool call shape, no newlines: <tool_call>NAME<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>. Tool results go in a <|Tool|> block; consecutive tool messages are merged.
• generation_config.json carries the base model's nonstandard max_tokens key and no max_new_tokens. HF generate() ignores it. Left verbatim on purpose — pass your own cap.

Known limits — please read before deploying

Raptor 0.6.1 preview is a practical checkpoint, and a preview: it is re-cut as the tune iterates. It was selected under an explicit tolerance for ordinary 4B-model mistakes. It is not a zero-failure model, and the following are measured or documented, not hypothetical.

One narrow weak spot: tool calls with reasoning turned off and no system prompt. When a conversation starts without tools and tools are offered on a later turn, the model can occasionally write a plausible-looking answer instead of calling the tool. Measured across a grid of reasoning lane x system prompt x two tool scenarios, n=12 samples per cell at the serving contract above:

Read that carefully, because the scope matters:

• Reasoning ON — this model's default — calls the tool every time. The weak cell only appears with reasoning explicitly disabled.
• Any system prompt that mentions using tools restores it to 12/12.
• It is scenario-specific, not a general tool-calling defect. A stock-price lookup in exactly the same conversational shape is 12/12 even in the weak cell; the weather prompt is the one where a strong pretrained "write a weather report" format prior competes with the tool call.
• The weakness pre-exists in the base model — the base is also imperfect in that same cell. Pooled across runs, base called the tool 31/48 versus 45/120 for the tuned weights, so the tune does measurably lower the rate in that one cell.
• It is not quantization damage: this bundle tracks its own bf16 source within measurement noise.
• No earlier training checkpoint is cleaner. Checkpoints u0002/u0004/u0006/u0008 were merged from the base and measured at n=24: 7, 12, 8 and 12 of 24. Two numerically near-identical builds of the same final checkpoint scored 12/24 and 6/24, which bounds the noise of this measurement and shows the trajectory carries no usable signal.

If you serve with reasoning off and rely on tools, put a line in your system prompt telling the model to call available tools. That is a complete mitigation in testing.

Also carried over from the checkpoint that produced these weights:

• Long-context recall is not guaranteed. A 64K-token retrieval prompt was answered correctly on a clarified retry, both with reasoning on and off, but an earlier long prompt failed to answer at all and instead copied and re-examined the source text. Allow clarification and retry rather than promising reliable completion of every long prompt.
• Ordinary agentic mistakes remain: stale or wrong tool names, schema and JSON errors, unsupported success claims, invented identifiers, dependent calls issued before their prerequisite, and arithmetic or recall slips. Do not execute generated mutations without your harness's normal approval step.
• Identity responses are not enforced. "Raptor" names the tuned artifact; the model will often still describe itself as Spark. Quality was preferred over persona enforcement.

Provenance

Quantized by Jinho Jang (eric@osaurus.ai). Base model XHToken/Spark-X2.5-4B, apache-2.0. Iteration 1 — see osaurus.json (model_version) for the bundle iteration and required_osaurus_version for the minimum runtime.