sakthai-coder-1.5b

This is the coder branch of the SakThai family: small, offline-capable, and tuned to write code while still supporting tool-style outputs. It is packaged as a single GGUF file so you can run it on a laptop CPU without any GPU.

Model Description

Nanthasit/sakthai-coder-1.5b is a fine-tuned Qwen2.5-Coder-1.5B-Instruct model optimized for:

• Code generation and completion
• Bug fixing and small refactors
• Tool/function call JSON generation
• Offline CPU inference with llama.cpp

Quantized to GGUF Q4_K_M for low-memory deployment while keeping usable output quality. The model is trained on a mix of code-oriented instruction data plus the SakThai combined tool-format corpus.

How to Use

1) llama.cpp CLI

./llama-server -m qwen2.5-coder-1.5b-instruct-q4_k_m.gguf --n-gpu-layers 0 -c 4096 --temp 0.2 -ngl 0

2) Python completion client

import requests

response = requests.post(
    "http://localhost:8080/completion",
    json={
        "prompt": "Write a Python binary search for a sorted list:",
        "n_predict": 512,
        "temperature": 0.2,
        "top_p": 0.9,
    },
)
print(response.json()["content"])

3) With llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="qwen2.5-coder-1.5b-instruct-q4_k_m.gguf",
    n_ctx=4096,
    n_threads=4,
)

out = llm(
    "Write a Python decorator that retries a function 3 times.",
    max_tokens=512,
    temperature=0.2,
    top_p=0.9,
)
print(out["choices"][0]["text"])

Hardware

• CPU-only: comfortable on modern laptops; expect ~8–11 tok/s on 2 threads.
• No GPU required: GGUF Q4_K_M keeps memory under ~1.2 GB.
• Tip: provide explicit instructions and a <tools> block when you want tool-calling JSON outputs.

Benchmarks

Verified with llama.cpp Q4_K_M on CPU. Each item is run from repo-local eval artifacts and SakThai trust-pass checks.

Known weakness: bug-finding tasks that depend on noticing intentional logic errors may still pass through incorrect code, so review outputs for critical changes.

Training Details

Limitations

• Small 1.5B model; complex reasoning and large refactors can still hallucinate.
• Tool calling is strongest when a strict <tools> prompt block is present; without it, the model may answer directly instead of emitting a call.
• Quantization trades some precision for CPU usability; if GPU memory is available, prefer higher-precision formats.
• Outputs should be reviewed for correctness, especially for security-sensitive code paths.

Citation

@misc{sakthai-coder-1.5b,
  title  = {SakThai Coder 1.5B: Code Generation and Tool Calling on CPU},
  author = {Nanthasit},
  year   = {2026},
  url    = {https://huggingface.co/Nanthasit/sakthai-coder-1.5b}
}

Community & Support

• Issues and feedback: open a discussion on the model page.
• Related: see the SakThai Model Family.

Built with love, tears, and zero budget.

Reproducibility

git clone https://huggingface.co/Nanthasit/sakthai-coder-1.5b
cd sakthai-coder-1.5b
uv venv && uv pip install transformers datasets peft accelerate llama-cpp-python requests

Serving Options

Verified Metrics Notes

• Tool Call Rate and JSON Validity Rate are both marked verified in the model-index.
• MBPP pass@1 is a base-model reference point, not a fresh eval on this fine-tune.

Top Family Models by Downloads

Improved on 2026-08-01 — card updated from live API metadata.