HRM-Text-1B-agent (v1) — tool / function calling
Full-parameter SFT of sapientinc/HRM-Text-1B — a 1B
base (pre-alignment) Hierarchical Reasoning Model — fine-tuned to do function / tool calling.
It takes a model that scored 0% on the task and turns it into a competent small tool-caller.
Code, full writeup, and the architecture experiments: https://github.com/jasoncarreira/hrm-text-agent
See also hrm-text-agent-v2, which adds xLAM parallel data for much stronger multi-call performance (with some tradeoffs).
Scores — BFCL v4 (official AST checker, full test sets)
| Category | n | Base | This model (v1) |
|---|---|---|---|
| simple | 400 | 0% | 61.5% |
| multiple | 200 | 0% | 53.5% |
| parallel | 200 | 0% | 37.5% |
| parallel_multiple | 200 | 0% | 28.0% |
| irrelevance | 240 | 100% | 80.8% |
Overall micro-average (1,240): 54.7%. Call-category competence (4 call cats, count-weighted): ~48%. This sits above generic 1B instruct models on BFCL non-live AST, and below the best purpose-built 1B (xLAM-2-1b-fc-r ~69) — strong for a 1B base + SFT.
General capability (base → tuned, 8-benchmark forgetting check)
The SFT was benign: reasoning/knowledge retained — GSM8k 84.5→85.6, DROP(F1) 84.8→83.3, and BoolQ/HellaSwag/Winogrande flat. The only drops were single-letter MCQ format discipline (MMLU 60.1→55.5, ARC-C 83.5→75.1), driven by a rise in non-letter outputs (~10% "invalid") — largely recoverable (the model often answers correctly in prose), and fixed in v2.
Training
Matches the sapientinc cfg_sft recipe:
- full-parameter SFT — not LoRA (LoRA on HRM's weight-shared recurrence amplifies the delta per-cycle and collapses the output distribution); bf16 autocast + fp32 master weights
- lr 3e-5, cosine decay to 10%, no warmup; AdamW (0.9, 0.95), weight_decay 0.1
- 3 epochs,
max_len2048, effective batch ~32; ~25k mixed examples, ~3.5 h on an A100 80GB - Uses the model's
directcondition token (<|object_ref_start|>) — the documented mode for structured output.
Data mix: tool calls (Hermes + glaive-function-calling) + general instructions (HuggingFaceH4/no_robots) + synthesized irrelevance ("tools present but none fit → don't call").
Usage
HRM-Text is a PrefixLM with a conditioning scheme: prompts open
<|im_start|><|object_ref_start|>…<|im_end|> with token_type_ids=1 over the prompt span. A naive
.generate() won't match the training distribution — use the agent loop / eval harness in the repo:
git clone https://github.com/jasoncarreira/hrm-text-agent && cd hrm-text-agent
pip install -r requirements.txt
python infer_agent.py --model jasoncarreira/hrm-text-agent "What's the weather in Paris?"
python bfcl_local.py --model jasoncarreira/hrm-text-agent --dump errs.jsonl # full BFCL
License & data lineage
The base model is Apache-2.0, but the training data includes HuggingFaceH4/no_robots (CC-BY-NC-4.0), so treat this derived model as research / non-commercial. Verify the licenses of all sources for your use case.
🤖 Built with Claude Code (including a second Claude driving training on the GPU pod).