Qwen3-0.6B-tools

Merged BF16 weights of the LoRA tool-calling fine-tune of Qwen/Qwen3-0.6B, trained so the model reliably emits OpenAI-style tool calls for agentic workflows.

Base model

This model was fine-tuned from:

Qwen/Qwen3-0.6B

Training

Supervised fine-tuning with LoRA, run with NeMo-RL, assistant-turn-masked cross-entropy over the tool-calling logs.

• LoRA: dim 32, alpha 32, dropout 0.05, target modules *.proj (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
• Max sequence length: 4096
• Precision: bfloat16
• Optimizer: AdamW, lr 5e-5, weight decay 0.01
• Schedule: 15-step linear warmup, then cosine decay to 1e-6
• Global batch size: 64 (micro batch 2)
• Training steps: 336 (validation every 84)
• Loss masked to assistant turns via the chat template's generation keyword

The tool-calling chat template is the model's contract: it is the template the base model's inline template was replaced with so that tool schemas render verbatim and the assistant's tool_calls block is the trainable span.

Dataset

Training used the sft_tools split of the r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation dataset (OpenAI messages format; assistant turns carry tool_calls).

Intended use

• Structured tool / function calling
• Agent-style multi-step interactions
• Small-footprint on-device or edge deployment

It is not intended to be a general replacement for larger Qwen models.

Sibling artifacts

• GGUF quantizations: iromu/Qwen3-0.6B-tools-GGUF
• NVFP4 checkpoint: iromu/Qwen3-0.6B-tools-NVFP4

Usage

Serve the NVFP4 checkpoint with TensorRT-LLM:

trtllm-serve serve iromu/Qwen3-0.6B-tools-NVFP4 --port 8000

Load a GGUF quant with llama.cpp:

llama-cli -hf iromu/Qwen3-0.6B-tools-GGUF:Q4_K_M

Load the merged BF16 model with transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "iromu/Qwen3-0.6B-tools", torch_dtype="bfloat16", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("iromu/Qwen3-0.6B-tools")

Validation

Measured on the sft_tools held-out validation split, greedy decoding. Throughput is single-stream greedy decode, not serving throughput.

suppressed in the Δ column means no comparison was admissible, which is a statement about the provenance of the two rows and not a measurement of no change. The reason is printed in the cell.