sakthai-context-0.5b-tools

SakThai Context 0.5B Tools is a prompt-masked SFT of Qwen2.5-0.5B-Instruct optimized for browser/tool calling. It achieves 91.2% selection accuracy on SakThai Bench v2, with 0% degenerate outputs in multi-trial evaluation.

Model Description

SakThai Context 0.5B Tools is a prompt-masked supervised fine-tune of Qwen/Qwen2.5-0.5B-Instruct focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.

Key points:

• Base: Qwen/Qwen2.5-0.5B-Instruct
• Training: prompt-masked SFT on tool-calling traces from Nanthasit/sakthai-combined-v7
• Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
• License: Apache-2.0

Quick Start — Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Nanthasit/sakthai-context-0.5b-tools"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {"location": {"type": "string"}},
                "required": ["location"],
            },
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What's the weather in Tokyo?"},
]

text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Quick Start — llama.cpp / Ollama

# Convert with llama.cpp and run locally
llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M
ollama create sakthai-context-0.5b-tools -f Modelfile
ollama run sakthai-context-0.5b-tools

Usage notes

• For tool calling, always use apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True) so the model receives the proper <tools> block.
• If you want stricter outputs, reduce temperature further, e.g. 0.0.
• For CPU-only inference, set device_map="cpu"; GPU/MPS/CPU auto-detection works with device_map="auto".

Architecture & Config

Benchmarks

Evidence: .eval_results/sakthai-bench-v2.yaml in repo.

Limitations

• 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
• Tool schema adherence degrades on nested arguments and long context traces.
• Current weights are merged; if you need the unmerged adapter, use Nanthasit/sakthai-context-0.5b-tools-sft or Nanthasit/sakthai-context-0.5b-tools-sft-v2.

Citation

If you use this model, please cite the SakThai model family and benchmark:

@misc{sakthai2025context05btools,
  title  = {SakThai Context 0.5B Tools},
  author = {Nanthasit},
  year   = {2026},
  url    = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}
}

SakThai Family

Download counts and sizes were verified from the Hub API at upload time.