Qwen2.5-1.5B-tools
Merged BF16 weights of the LoRA tool-calling fine-tune of
unsloth/Qwen2.5-1.5B-Instruct. Built by
scripts/build_tools_gguf.sh Qwen2.5-1.5B-tools (merge_and_unload of the
latest checkpoint in checkpoints/Qwen2.5-1.5B-tools/).
Training
- Recipe:
TrainFinetuneRecipeForNextTokenPrediction(NeMo AutoModel 0.5.0) - Config:
configs/qwen25_1.5b_tools.yaml - LoRA: dim 32, alpha 32, dropout 0.05, target_modules
*.proj - Data:
sft_toolssplit ofr0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation - Learning rate: 1e-5 (cosine decay), 501 steps, global batch 64, seq length 4096
Sibling artifacts
- GGUF quantizations:
models/Qwen2.5-1.5B-tools-GGUF/ - NVFP4 checkpoint:
models/Qwen2.5-1.5B-tools-NVFP4/
Validation matrix
Tool-calling validation on the sft_tools validation split (greedy decoding, 384 max new tokens). Throughput is single-stream greedy decode, not serving throughput.
Pretrained base (unsloth/Qwen2.5-1.5B-Instruct): 8.8% exact-args match (24/274). Fine-tuned (BF16): 0.7% exact-args match (2/274) (-8pp vs base).
- GGUF-BF16: 2/274 (0.7%) exact, 88.6 tok/s — 100% of BF16.
- GGUF-Q4_K_M: 3/274 (1.1%) exact, 142.9 tok/s — 150% of BF16.
- GGUF-Q5_K_M: 6/274 (2.2%) exact, 131.4 tok/s — 300% of BF16.
- GGUF-Q8_0: 2/274 (0.7%) exact, 115.3 tok/s — 100% of BF16.
| Model | Quant | n | Tool call emitted | Names match | Exact args match | Δ exact vs BASE | tok/s |
|---|---|---|---|---|---|---|---|
| Qwen2.5-1.5B-tools | BASE (unsloth/Qwen2.5-1.5B-Instruct) | 274 | 272/274 (99.3%) | 59/274 (21.5%) | 24/274 (8.8%) | — | 40.6 |
| Qwen2.5-1.5B-tools | BF16 | 274 | 258/274 (94.2%) | 216/274 (78.8%) | 2/274 (0.7%) | -8pp | 38.0 |
| Qwen2.5-1.5B-tools | GGUF-BF16 | 274 | 272/274 (99.3%) | 167/274 (60.9%) | 2/274 (0.7%) | -8pp | 88.6 |
| Qwen2.5-1.5B-tools | GGUF-Q4_K_M | 274 | 248/274 (90.5%) | 49/274 (17.9%) | 3/274 (1.1%) | -7.7pp | 142.9 |
| Qwen2.5-1.5B-tools | GGUF-Q5_K_M | 274 | 260/274 (94.9%) | 79/274 (28.8%) | 6/274 (2.2%) | -6.6pp | 131.4 |
| Qwen2.5-1.5B-tools | GGUF-Q8_0 | 274 | 271/274 (98.9%) | 187/274 (68.2%) | 2/274 (0.7%) | -8pp | 115.3 |