TOAM-100M v0.2

TOAM-100M is a 102M-parameter model for tool use and agent control. You give it tool schemas and a request, and it returns the next action as JSON. It is trained from scratch (not fine-tuned from another model) and loads as a standard LlamaForCausalLM.

Compared with FunctionGemma-270M, on the same harness and GPU (details):

• 38 % of its parameters
• about the same BFCL Live accuracy (26.2 vs 26.6 %)
• 3.5× lower median latency per request (207 vs 731 ms)
• but only 64 % of its BFCL Non-live accuracy (28.0 vs 43.6 %), mostly because of requests that need several calls at once. See limitations.

Research release. TOAM is shared as a research artefact and proof of work. It explores how much tool-use and agent-control ability a very small model trained from scratch can achieve, and documents the process openly so others can learn from it, reproduce it and build on it. It is not a state-of-the-art model and has not been validated for production use. Please do not use it for safety-critical, medical, legal, financial or other high-stakes decisions, or let it trigger irreversible actions without human review. Treat its outputs as suggestions, and validate every tool call before it runs. Feedback and failure reports are very welcome and directly shape future versions.

What changed since v0.1

v0.2 finishes the pretraining run and improves the fine-tuning data: minimal-pair examples that teach when to call vs. ask vs. confirm, cleaner generated requests, and about 1,800 public training labels removed after an audit found many of them asked the user for values the request already contained.

Example

tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Get the current weather for a city.",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]

agent.decide(tools, "What's the weather in Pune right now?")

{"type": "tool_call", "tool": "get_weather", "arguments": {"city": "Pune"}}

The model can also call several tools at once, ask the user for a missing value or for confirmation, say that no tool fits, or escalate.

Why TOAM?

A general-purpose LLM spends much of its capacity storing knowledge about the world. An agent controller can get the knowledge it needs at run time from tool schemas, tool results, state and context. TOAM tests how small a model can be when its main job is to decide the next action, not to know everything itself. The goal is useful capability per unit of memory, latency and energy, not the highest benchmark score.

Intended use

• Picking a tool and filling its arguments in a small or local agent
• Routing requests in an agent pipeline (call, ask, confirm, refuse, escalate)
• Embedded, on-device or CPU-only agent controllers where a large model is too heavy
• Research on small agentic language models

Out of scope: TOAM-100M is not a chatbot, assistant or knowledge model. It knows little about the world, writes poor free text, and should not answer questions or be used without a tool menu.

Usage

TOAM uses its own raw prompt and action format. It is not a chat model: do not apply a chat template (there is none, and a generic one will give garbage). tiny_agent.py in this repository builds the prompt, runs greedy decoding and parses the action:

from tiny_agent import TinyAgent

agent = TinyAgent("ksg91/toam-100m-v0.2")
agent.decide(tools, "What's the weather in Pune right now?")
# {'type': 'tool_call', 'tool': 'get_weather', 'arguments': {'city': 'Pune'}}

Optional inputs: policy (plain-text rules, e.g. "Confirm with the user before any booking or payment."), history (earlier tool calls and their results), state (e.g. {"today": "2026-10-02"} so relative dates resolve), and a multi-turn conversation as the goal ("User: ...\nAssistant: ...\nUser: ...").

Actions: tool_call, tool_calls (parallel), ask_user (a required value is missing, confirmation, or no tool fits), escalate, and the multi-step control actions finish / retry / wait.

The context window is 2,048 tokens, so keep menus to about 10 tools with short descriptions. The prompt and action format is unchanged from v0.1.

Raw prompt format

This is what tiny_agent.format_prompt(...) produces for the example above. The model writes the action after <|action|> and then stops at EOS:

<|system|>
You are an agent controller. Choose the correct next action.
<|tools|>
[{"name":"get_weather","description":"Get the current weather for a city.","parameters":{"properties":{"city":{"type":"string"}},"required":["city"]}}]
<|goal|>
What's the weather in Pune right now?
<|history|>
[]
<|state|>
{"step":0}
<|observation|>
null
<|action|>

The <|...|> tags are special tokens and must be tokenized as such. Always build the tool list with format_prompt, because it renders schemas the way the model saw them in training.

GGUF (llama.cpp, LM Studio, Ollama)

Send the prompt from format_prompt(...) as raw text (a completion, not a chat) with greedy decoding. llama-server parses the special tokens by default:

llama-server -m toam-100m-v0.2-q8_0.gguf -c 2048
curl http://127.0.0.1:8080/completion -d '{"prompt": "<prompt from format_prompt>", "temperature": 0, "n_predict": 256}'

Equivalence check (llama.cpp build b11223, against the transformers model): identical tokenization on 39/39 prompts, and identical greedy outputs on 35/35 held-out prompts for both F16 and Q8_0. On CPU only (16 threads, llama-server), a request takes about 280 ms (F16) to 330 ms (Q8_0) at the median.

Weights

model.safetensors is stored in FP16 (205 MB). Training used FP32 master weights. On 100 held-out prompts, FP16 gives the same greedy outputs as FP32 (checked on CPU). tiny_agent.py runs in BF16 on CUDA and FP32 on CPU by default; pass dtype="float32" for exact FP32 behaviour.

Benchmarks

1. Independent benchmark: BFCL v4, single-turn

BFCL prompts and functions were not written by us. Fine-tuning data was decontaminated against BFCL. The pretraining corpus was not (see Training). All models were run locally with the same harness and scored with bfcl_eval's own AST checker. This is not an official leaderboard submission. Our FunctionGemma-270M numbers agree with the ones on its model card, which is a check that the harness is correct.

By category (TOAM v0.2 / FunctionGemma-270M): simple Python 46.8 / 62.7, multiple 51.5 / 59.0, parallel 16.0 / 36.0, parallel multiple 7.5 / 31.0, live simple 33.7 / 32.9, live multiple 24.9 / 25.3, live irrelevance 76.0 / 80.2. Irrelevance means the model correctly makes no call when no tool fits. 38 of the ~3,600 prompts are longer than TOAM's 2,048-token context and count as failures.

Benchmark environment

Latency depends heavily on the inference stack. These numbers compare models in one harness, not optimised deployments. TOAM's peak VRAM is higher than its size suggests because the benchmark kept FP32 weights under BF16 autocast.

2. Internal held-out evaluation

These tests use tools and services that never appeared in any training data. For the held-out servers that includes pretraining. The tests are in-house, and the requests look more like the training data than BFCL's do.

The SGD dialogue format itself (for other services) was in training.

3. Close to the training distribution

These are new rows, but they come from the same generators or sources as the training data. Read them as a sanity check, not as a measure of generalisation.

The public-data slice is lower than in v0.1 (73.9 %) only on When2Call's "ask the user" rows (1.00 → 0.38): v0.2 was deliberately not trained on those labels after an audit found many of them ask for values the request already contains. Every other public source scores the same as before.

Limitations

• Several calls in one request are the main weakness (BFCL parallel 16 %, parallel multiple 7.5 %). On long multi-part requests the model can lose track: it may drop a sub-task, or change a value it should keep when repeating a call ("5 Tesla … then repeat with 10 seconds" can also change the field to 10).
• Binding values to the right field still fails sometimes when a request has several values (e.g. putting an artist name into genre and dropping the year).
• Weak on maths/science-style functions, which make up much of BFCL Non-live.
• The questions it asks can be clumsy.
• English only. Context is 2,048 tokens.
• Always validate the returned arguments before running a tool, and require confirmation for anything irreversible.

Training

• Architecture: Llama-style decoder (RMSNorm, SwiGLU, RoPE, grouped-query attention with 12 query / 4 KV heads, d=768, 12 layers, tied embeddings), 32k byte-level BPE tokenizer, 2,048-token context. 102.5M parameters. Trained from scratch.
• Pretraining: 10B tokens (warmup-stable-decay schedule) sampled from a 19.6B-token corpus of public data: web and educational text (roughly 60 % of what the model saw), code (about 25 %), structured data and API specifications (JSON/YAML, OpenAPI, JSON Schema; about 12 %) and encyclopedic text (about 5 %). The corpus was decontaminated against the 84 held-out tool servers (matching tool names, identifiers and description shingles) before tokenization. It was not decontaminated against BFCL.
• Fine-tuning: one epoch on about 137k rows. About 55 % are in-house data generated from real-world MCP tool schemas (single-step requests, minimal pairs that differ in one detail, and multi-step episodes in simulated tool worlds). About 45 % are public function-calling and human-written dialogue/intent data, with 25 % typo/noise augmentation. Every label is checked automatically: calls must be schema-valid, and every argument must be traceable to the request, the state or a schema default (otherwise the right action is to ask the user). Rows overlapping BFCL (function names, 8-gram overlap) or the held-out tools are removed.

The in-house training data, its generators and the exact mixture are not released.

Versions

Each checkpoint is published as its own repository, and existing weights are never replaced.

Next: better data for multi-call requests and scientific/standard-library style functions, and more natural generated requests.

Acknowledgements

Fine-tuning data includes openly licensed datasets from Salesforce and Argilla (APIGen / xLAM), Team-ACE (ToolACE), NVIDIA (When2Call), Google (Schema-Guided Dialogue, CC BY-SA 4.0) and Amazon (MASSIVE, CC BY 4.0). Pretraining data includes FineWeb-Edu, Cosmopedia (SmolLM corpus), CodeParrot, StarCoderData, CodeSearchNet, Wikipedia, the APIs.guru OpenAPI directory and SchemaStore. Evaluation uses the Berkeley Function-Calling Leaderboard (BFCL v4).

License

Apache-2.0 (weights and inference code). The training datasets keep their own licenses.