LFM2.5-2.6B ThinkingCap Distiller

⚠️ This is NOT a chat model. This is a trace-processing tool. Its job is to take a long reasoning trace (multi-turn agentic rollout, tool calls, verbose reasoning) and produce a condensed ThinkingCap-style trace: a short <think> + a definitive numbered answer. Feed it traces, not conversation. The output is meant to be parsed and served as training data, not read as a chatbot reply.

A compact reasoning-trace translator that re-does verbose frontier-model rollouts in the ThinkingCap (TC) style: a short <think> block plus a definitive, numbered answer — no meta-commentary, no loops, no wasted tokens.

Built by distilling 1,985 curated traces generated by bottlecapai/ThinkingCap-Qwen3.6-27B over a multi-teacher corpus (Qwen3.8-Max, GLM-5.2, Kimi K3) from r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation, then fine-tuned into LiquidAI/LFM2.5-2.6B via LoRA.

TL;DR — a ~2.6B local model that turns a 10K-char reasoning trace into a ~1.3K-char condensed TC trace in ~4 s on an iGPU (median 3.6x compression, up to 88x), preserving the verdict and the numbers — matching a 27B teacher on ground-truth verdicts (90% vs 90%).

Why this model exists — the distillation vision

New frontier models keep appearing, each with its own expert strengths (coding, math, tool-use, legal reasoning…) expressed as long reasoning traces. This model exists to make that knowledge portable:

• Expertise capture: feed it any trace from any model → it extracts the expert reasoning (the essential steps, calculations, verdicts) into a compact, reusable TC trace.
• Native ThinkingCap on any model: those condensed traces can train a new student model to think in TC style natively — its <think> becomes token-efficient, and its answers become decisive.
• Task-specific dataset factory: run it over traces from your target domain → you get a ready-to-use SFT dataset for that specific task, for any base model you choose.
• Self-distillation: use it on traces produced by the same model that runs it — condense your own rollouts, retrain on the condensed output, iterate. A closed loop that keeps getting cheaper and sharper.

In short: traces in → expertise + native thinking style out, on demand, locally, for any domain and any target model.

Why this base model

LFM2.5-2.6B was chosen over similarly-sized alternatives (e.g. Qwen3.5-4B, gemma-4-E2B) because a trace distiller lives or dies by instruction following: it must obey a strict output contract (<think>…</think> + numbered verdict), preserve exact literals, and never add extra steps. In that dimension LFM2.5-2.6B is exceptional for its size — it leads or ties models 2-4x larger on instruction-following (IFBench, Multi-IF, IFStruct), tool-use (BFCLv4, ToolSandbox) and non-hallucination (AA-Omni Non-hallu) benchmarks, while staying small enough to run on an iGPU. That is exactly the capability a trace condenser must not lose.

What gets condensed

The source traces are multi-step, multi-turn agentic rollouts: a teacher model (Qwen3.8-Max / GLM-5.2 / Kimi K3) executes a task through repeated [assistant] turns, tool calls, tool results, corrections, and verbose self-talk — often with dead ends and repeated attempts. The distiller collapses the whole multi-turn rollout into a single condensed trace: one short <think> (the essential reasoning) + one answer (the verified outcome), keeping the task-type structure (atomic / chained / decomposed / staged / tool-use) visible.

Measured reduction (1,985 training traces):

Quick start

Python (transformers 5.x)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "osk-arr00/lfm2.5-2.6B-thinkingcap-distiller"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

system_prompt = SYSTEM_PROMPT  # see section below

trace = ""  # the full teacher trace, rendered with [system]/[user]/[assistant] markers

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": trace},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=16384).to(model.device)

with torch.inference_mode():
    out = model.generate(
        inputs["input_ids"],
        max_new_tokens=2048,
        do_sample=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

generated = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

llama.cpp — Strix Halo / RADV (recommended)

⚠️ Use the Vulkan RADV performance build of llama.cpp. The kyuz0/amd-strix-halo-toolboxes:vulkan-radv-performance Docker image ships a llama.cpp build tuned for AMD Strix Halo iGPUs (Radeon 8000 series, gfx1151), including --kv-unified support required by the UMA architecture. See the toolbox repo.

docker run -d --name llama-mini-dest \
  --device /dev/dri:/dev/dri \
  --group-add 983 --group-add 987 \
  -v "$PWD:/models" \
  --ulimit memlock=-1 \
  -p 8080:8080 \
  kyuz0/amd-strix-halo-toolboxes:vulkan-radv-performance \
  llama-server --host 0.0.0.0 --port 8080 \
  -m /models/lfm25-26b-thinkingcap-distiller-q4_k_m.gguf \
  -ngl 999 --flash-attn on --jinja \
  --ctx-size 131072 \
  -ctk q8_0 -ctv q8_0 -b 8192 -ub 4096 \
  -t 8 -tb 16 --no-context-shift \
  --kv-unified -np 4

Query it:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "<system prompt>"},
      {"role": "user", "content": "<trace>"}
    ],
    "max_tokens": 2048,
    "temperature": 0.0
  }'

System prompt

The translator is instruction-driven. Use the same prompt that generated the training data (tc_system_prompt.txt in this repo). Its anatomy:

1. Role — "reasoning-trace translator": translate, analyze, condense.
2. Format — write everything in the final answer: <think>…</think> + numbered steps + definitive verdict.
3. Anti-meta rule — never mention the trace or the translation process; think about the task, from scratch.
4. Task-type guidance — atomic, chained, decomposition/planning, staged elaboration, and tool-use tasks, each with its handling.

Full text: tc_system_prompt.txt.

Data format

Input: the complete teacher trace as the user message, rendered with [system] / [user] / [assistant] / [assistant tool_call] / [tool] markers (same rendering used for training):

[system]
<teacher system prompt>
[user]
<task prompt>
[assistant]
<verbose rollout>
[assistant tool_call]
{"name": "corpus_search", "arguments": "..."}
[tool]
<tool result>
[assistant]
<more rollout>

Parsing the output → building your own dataset

The model is designed to feed a data pipeline. After generation, split the raw output into its three components and serve them as a training row for any student model:

def parse_output(raw: str):
    """Split the model output into (pseudo_think, answer).

    The LFM chat template already opens <think> for us, so the generated
    text usually starts with the thinking content and closes it with
    </think>. Sometimes the model emits <think> itself — handle both.
    """
    if "</think>" in raw:
        head = raw.split("</think>", 1)[0]
        think = head.split("<think>", 1)[-1].strip()   # strip explicit <think> if present
        answer = raw.split("</think>", 1)[1].strip()
    else:
        think, answer = "", raw.strip()
    return think, answer

def make_training_row(user_prompt: str, trace: str, raw_output: str,
                      trace_id: str, system_prompt: str = SYSTEM_PROMPT):
    """Assemble a ready-to-train row for a chat-templated student model."""
    think, answer = parse_output(raw_output)
    return {
        # what the student should SEE (system + the original task prompt)
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},   # original task prompt only
        ],
        # what the student should LEARN to generate
        "target": {
            "thinking": think,     # condensed pseudo-think  (may be empty)
            "answer":   answer,    # the definitive verdict
        },
        # provenance for auditing
        "meta": {
            "source_trace_id": trace_id,
            "source_trace_chars": len(trace),
            "compression_x": round((len(trace) / max(1, len(think) + len(answer))), 1),
        },
    }

Key points:

• Separate the user prompt from the trace: the original task prompt (the [user] block of the trace) is the student's input; the condensed output is the student's target. The teacher's verbose rollout is reference material — it does not go into the student's training row.
• Pseudo-think ≠ the model's hidden thinking: the <think> block in the output is generated text — you can train the student to emit it verbatim, or map it to the student's native reasoning_content field.
• Answer = ground truth: the part after </think> is the verified verdict; it becomes the student's final content.
• Feed it to any template: ChatML, reasoning_content, or your own — the split gives you the three fields you need.

Worked example

Input — prime-number puzzle (10,767 chars, math domain):

[system]
You are an expert mathematician. Solve the given problem with complete rigor.
[user]
Moving a decimal point of a prime number to the right by one place makes it 5.4 more
than the original value. What was the original prime number?
[assistant]
<think>Let me denote the original prime number as p. Moving the decimal point one
place to the right means multiplying by 10. So the new number is 10p. The condition
states that ... 10p = p + 5.4 ... p = 0.6. But 0.6 is not a prime number. Let me
reconsider the problem. Wait, maybe "moving a decimal point to the right" means
inserting a decimal point into the number ... For example, if the number is 67,
placing a decimal point gives 6.7 ... Hmm, but that gives a larger number ...
</think> ...

Output — condensed TC trace (920 chars):

<think>
Let p be the number. Moving the decimal point one place to the right multiplies
the value by 10. The new value is 10p. The condition is: 10p = p + 5.4. 9p = 5.4,
p = 0.6. Check: 6 - 0.6 = 5.4. Constraint: p must be prime. 0.6 is not prime.
However, this is the unique arithmetic solution. The term "prime" is likely an
error in the problem statement or a mistranslation. I will provide the calculated value.
</think>
1. Let the original number be p.
2. Moving the decimal point one place to the right is equivalent to multiplying by 10.
3. Set up the equation based on the difference: 10p - p = 5.4 ⟹ 9p = 5.4 ⟹ p = 0.6.
4. Verification: 6 - 0.6 = 5.4. Note: 0.6 is not technically a prime number, but it
   is the unique solution satisfying the arithmetic conditions of the problem.
The original number is 0.6.

💡 10,767 → 920 chars = ~11.7x compression, with the verdict and the primality caveat preserved.

How it was built

A three-stage pipeline:

1. Curation — sample 1,991 traces from the 44,151-row multi-teacher corpus (r0b0tlab/…-distillation), stratified across 8 domains (math, code, reasoning, agent tool-use, tool recovery, strict instruction, iterative instruction, stateful dialogue). Each trace is rendered with [system]/[user]/[assistant]/[assistant tool_call]/[tool] markers so the full multi-turn rollout survives.
2. Distillation — feed each trace to the ThinkingCap teacher (bottlecapai/ThinkingCap-Qwen3.6-27B, --reasoning on) with a hand-tuned system prompt (see above) that instructs the teacher to re-do the reasoning in condensed TC style and output <think>…</think> + numbered verdict. A mini agnostic worked example in the prompt eliminated teacher meta-commentary (53-64% → 0%). The teacher's native reasoning_content is ignored; the formatted answer is the dataset target.
3. Clean + fine-tune — remove residual markdown decoration where non-structural (0 numeric alterations), drop 6 corrupt rows, then LoRA fine-tune LFM2.5-2.6B (r=16 on all linear projections + lm_head, bf16, max_seq_length=16384 so the 6% of traces >8K tokens are kept, LR 3e-4, 2 epochs, grad-checkpointing with activation offload) on an A100-40GB (~41 min).

At inference the model reproduces the same behavior: its LFM chat template opens <think>, the model plans there, closes it, and the answer after </think> is the condensed trace.

Training data

• Source: osk-arr00/thinkingcap-condensed-qwen3.8-glm5.2-kimi-k3 (our published dataset) built from r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation — a quality-filtered, deduplicated, multi-teacher SFT corpus (Qwen3.8-Max, GLM-5.2, Kimi K3) covering math, code, reasoning, instruction-following, tool-use, science, long-context and dialogue. 44,151 rows sampled → 1,991 curated → 1,985 after cleaning.
• Teacher for condensation: bottlecapai/ThinkingCap-Qwen3.6-27B (Apache-2.0), served with --reasoning on.
• Domain mix (condensed set): iterative_instruction (194), executed_tool_recovery (395), agent_tool (402), strict_instruction (300), math (250), code (200), reasoning (150), stateful_dialogue (100).
• Cleaning: prompts recovered from source; 6 corrupt rows (broken <think> parse) removed; markdown decoration removed where non-structural (**, bullets) with 0 numeric alterations; list bullets preserved where structural.

Evaluation

Setup: 100 held-out traces (not in training), q4_K_M GGUF via llama.cpp (Vulkan RADV on AMD Radeon 8050S / gfx1151, 131K ctx, 4 parallel slots).

Ground-truth comparison (vs the source teacher)

The strongest signal: does the condensed trace solve the task — i.e. does its final verdict match the source teacher's verified final answer (the last assistant turn of the original trace, used as ground truth)?

* math breakdown (7 nominal misses): 4 are extraction false-negatives (equivalent answers, e.g. both conclude b=7), 1 truncation (generation hit max_new_tokens), 1 case where the mini is actually more correct than TC (TC computed 4^8=65536 from a √2 factor; mini computed (√2)^8=16), 1 borderline. Real error rate vs TC: ~1/10 in math.

Bottom line: the 2.6B distiller matches the 27B teacher on verifiable verdicts (90% vs 90%), beats it on strict_instruction (+27 pts) and executed_tool_recovery (+13 pts), and trails slightly on math.

⚠️ Automatic metrics (token overlap, last-number match, phrase match) are misleading for this task: they penalize legitimate format variation (e.g. "batch_size is now safe" vs "The batch_size value is now safe"). Ground-truth verdict comparison + manual review are the ground truth.

Limitations

• Small model: 2.6B — complex multi-step reasoning degrades vs. the 27B teacher; traces >16K tokens are truncated (feed them in chunks).
• Verbatim fidelity: with temperature=0 the distiller is more faithful than the teacher on strict JSON tasks (94% vs 67% GT hits), but exact literal ids can still drift on long outputs — verify in strict pipelines.
• Languages: trained mostly on English; Spanish/others come from the base model prior, not from this fine-tune.
• Truncation: traces >16K tokens are truncated (feed them in chunks).
• No QAT yet: the q4 is post-training quantization; a QAT pass is planned to recover the small remaining loss.

License & credits

Model license: Apache 2.0. Note: the base model LiquidAI/LFM2.5-2.6B is under the LFM Open License v1.0 (commercial use permitted for entities below a $10M/yr revenue threshold — see Section 5 of the license). Review its terms before commercial deployment of derived models.

Dataset license: Apache-2.0 — see the published dataset osk-arr00/thinkingcap-condensed-qwen3.8-glm5.2-kimi-k3. The upstream corpus labels its rows other; synthetic research corpus with per-row source_license provenance; we rely on that classification.

Credits

Citation

@misc{lfm25-thinkingcap-distiller,
  title  = {LFM2.5-2.6B ThinkingCap Distiller: a condensed reasoning-trace translator},
  author = {Oscar},
  year   = {2026},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/osk-arr00/lfm2.5-2.6B-thinkingcap-distiller}
}