OsaurusAI/Raptor-0.6.1-preview-4B-JANG_6M

Raptor 0.6.1 preview — the next Raptor tune on Spark-X2.5-4B, trained on Osaurus harness decisions, quantized to 3.41 GiB for Apple Silicon.

This is a preview. The released model is OsaurusAI/Raptor-0.6-4B-JANG_6M; this repo is the candidate that may replace it, published for testing.

Raptor 0.6.1 is a supervised fine-tune of XHToken/Spark-X2.5-4B (4.112B dense, reasoning + tool use), built for life inside Osaurus: when to call a tool and when to just answer, which tool to use, how to recover from errors, when to ask, and how to hand work to sub-agents — with reasoning on (<think>) and off. Tokenizer and chat template are unchanged from the base.

Same JANG_6M layout as Raptor 0.6: attention and the tied embedding at 8-bit, the MLP at 6-bit, group size 64, every group scale in bfloat16, and the per-head attention gate and all norms kept in full precision.

What changed vs Raptor 0.6

• A wider tune. LoRA SFT (rank 16, α 32) on attention and MLP — 180 matrices, merged into the weights — where 0.6 touched attention only.
• A harness-built corpus. ~3.5k decisions: on-policy self-distilled answers (the base model's own accepted replies replace hand-written ones where it already gets them right), plus targeted batches for the weaknesses found in 0.6 — MCP ambiguity, undo/recovery, answer-vs-ask, scheduling, legacy tool discovery, act-don't-claim, acting despite history, confirm-before-destructive, and no-folder onboarding.
• Per-decision loss (every decision weighs the same regardless of length), early checkpoint (6 of 16 updates) — the dose that kept the gains without the fabrication rise seen later.

Behaviour vs Raptor 0.6 on the Osaurus harness suite (3 seeds per model, seed-averaged per case, bootstrap 95% CI, Bonferroni per category):

Quality of this quantization — measured, not estimated

KL divergence (nats per next-token position) and top-1 agreement against the model's own bf16 weights, full-sequence forward on two held-out sets that were never in calibration.

Calibration cuts the KL 2.6–2.9× at identical size and speed; the gain is largest on harness conversations (reasoning-on −69%, reasoning-off −62%). These sets are long reasoning and tool traces and are harder per token than the set on the 0.6 card, so the numbers are not comparable with that card's.

Median of 4 probes at a fixed condition (512-token prompt, 128 generated), first probe discarded, M5 Max; the two bundles are within measurement spread of each other.

There is no stock-MLX comparison row, and that is not an omission. mlx-lm has no spark2_5 architecture, so no stock MLX quantization of this model exists to score against; the uncalibrated-rounding column above is the closest equivalent. The MLX runtime ships with the Osaurus harness.

Calibration

1.69M tokens captured on these tuned weights (the tune moved attention and MLP, so 0.6's statistics were not reused), taken from Osaurus harness conversations — system prompt + real Osaurus tool schemas, multi-turn, tool calls and grouped tool results, errors and recovery, no-folder turns — with reasoning on and off, plus the model's own generations on harness, code, tool-call and general prompts. One capture drives activation-aware scaling (72/72 norm fold sites, 36/36 attention gate projections), per-channel importance, and the error-corrected fit (180/181 tensors; the holdout is the tied embedding, a lookup with no input activation to condition on).

The capture has no Chinese, academic multiple-choice or >4K-token text. The Chinese-answer behaviour check passes, but quantization fidelity on those slices is not separately measured.

Running it in Osaurus

Requires Osaurus 0.25.14 or newer (declared in osaurus.json). Point Osaurus at the repo and it serves with the sampling, reasoning and tool-call contract below already declared in the bundle.

No working folder attached: the model does not reliably learn the "please add a folder" reply from training alone. When no folder is attached, append this to the system prompt — on 12 fresh requests it produced the right reply 96% of the time with reasoning on and 85% with reasoning off, versus 0% without it:

Workspace status: no folder is attached to this chat, so you cannot see or change any files on the user's Mac and no file tools exist. If the user asks for anything involving their files, folders, code or documents, do not call a tool: reply that no folder is attached and tell them to click the + button in this chat and add the folder that contains it. Offer to help as soon as it is added.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("OsaurusAI/Raptor-0.6.1-preview-4B-JANG_6M")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is 84 * 3 / 2?"}],
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

A bare mlx_lm.load will not resolve spark2_5 on its own — the runtime ships with the harness.

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=-1, no repetition penalty — carried in both generation_config.json and jang_config.json.
• Reasoning is ON by default. The generation prompt ends with <|Bot|><think>; with enable_thinking=false it ends <|Bot|></think>. A reasoning parser that waits for a literal <think> in the output stream will never see one.
• Give reasoning room. Traces on hard prompts can exceed 8,192 output tokens.
• Tools render into the first system block (## Tools plus a <tools> JSON list), so a tool-set change is a full re-prefill, not a suffix append.
• Tool call shape, no newlines: <tool_call>NAME<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>. Several independent calls may be emitted in one message; Osaurus runs them in parallel. Tool results go in a <|Tool|> block; consecutive tool messages are merged.
• generation_config.json carries the base model's nonstandard max_tokens key and no max_new_tokens; pass your own cap.

Known limits — please read before deploying

This is a preview of a 4B agent model, not a zero-failure model. Measured against Raptor 0.6 (3 seeds):

• Multi-step tasks: it finishes early without the required action more often (4.5% vs 2.1%).
• Safety approvals: it runs a forbidden tool without approval slightly more often (27/160 vs 17/140 pooled). Keep your harness's approval step for mutations.
• Outside a harness, keep a system prompt. With reasoning off and no system prompt at all, a tool offered mid-conversation can get a written answer instead of a call (4/12 sampled runs call correctly — 3/12 on this model's own bf16 weights, so it is not quantization damage). A one-line instruction to call available tools restores it (11/12); with reasoning on it is 12/12.
• Ordinary 4B mistakes remain: wrong tool names, schema and JSON slips, unsupported success claims, invented identifiers, and arithmetic or recall errors.
• Identity responses are not enforced; the model may describe itself as Spark.

Provenance

Quantized by Jinho Jang (eric@osaurus.ai). Base model XHToken/Spark-X2.5-4B (rev 5e10fcc), apache-2.0. Iteration 1 — see osaurus.json (model_version) for the bundle iteration and required_osaurus_version for the minimum runtime.

License

Raptor 0.6.1 preview is a derivative of XHToken/Spark-X2.5-4B by the SparkLLM Team, released under the Apache License, Version 2.0. The full license text is in the LICENSE file, carried over unchanged from the base model.

This model is distributed under the same Apache License, Version 2.0. In accordance with Section 4 of that license, the changes made to the original work are: (1) a supervised fine-tune of the attention and MLP projections (rank-16 LoRA on 180 matrices, merged into the weights; embeddings, norms and the attention gate unchanged), and (2) quantization of the merged weights to the JANG_6M format (affine 6-bit MLP, 8-bit attention and embedding, group size 64) with AWQ, GPTQ and importance-matrix calibration. The tokenizer, chat template and generation defaults are unchanged from the base model. Osaurus, Raptor and the Osaurus banner are trademarks of Osaurus and are not covered by the Apache license.

If you use this model, please also credit the base model:

@misc{sparkx2.5,
    title  = {Spark-X2.5 4B&1.7B: Pushing the Limits of Agentic Capabilities in On-Device Models},
    author = {SparkLLM Team},
    year   = {2026}
}