Qwen3.5-4B-InstructTool 🛠️

zinzinmit/Qwen3.5-4B-InstructTool is a specialized 4.2B parameter tool-calling and multi-turn planning language model, fine-tuned from Qwen/Qwen3.5-4B for the LifeMate / Zalo AI Challenge.

It is trained specifically for complex Vietnamese conversational agent workflows, multi-step environment manipulation, dynamic few-shot retrieval, schema-compliant JSON tool calling, and strict non-redundant environment interactions across 46 system and application tools.

🏆 Benchmark & Evaluation Results

Evaluated on the standardized LifeMate Validation Benchmark (data/train+public_test/benchmark/llm/val.jsonl, 155 multi-turn dialogue sessions with exact environment state verification):

Comparison with Base Models

Key Improvements over Base Qwen3.5-4B:

• +34.13% absolute increase in Official Competition Score (from 44.11% $\rightarrow$ 78.24%).
• +23.23% increase in Environment State Hash Accuracy (from 61.29% $\rightarrow$ 84.52%; 131/155 test environments in the exact expected final state).
• +39.35% increase in Call Exact Match (from 32.26% $\rightarrow$ 71.61%; 111/155 dialogues had 100% exact tool calls and arguments with 0 errors).
• Solved Zero-Call Over-Triggering: Zero-call accuracy jumped from 9.09% $\rightarrow$ 90.91%, eliminating redundant exploratory tool calls when the user query is chitchat or out-of-scope.
• Fast Serving with vLLM: Average end-to-end multi-turn dialogue inference is only 1.21s / session.

⚙️ Model Architecture & Technical Details

• Base Architecture: Qwen/Qwen3.5-4B (hybrid Gated-DeltaNet linear attention + grouped-query attention).
• Total Parameters: 4,205,751,296 (~4.21B parameters).
• Context Length: Native 16,384 tokens (16K).
• Training Method: PEFT LoRA ($r=16, \alpha=32$, targeting all linear projections q, k, v, o, gate, up, down, in_proj_qkv, in_proj_z, out_proj), merged back into native 16-bit bfloat16 safetensors.
• Dataset: LifeMate multi-turn dialogue benchmark (1,390 dialogues expanded into 3,759 causal step-level execution pairs).
• Competition Rule Compliance: Total parameters (4.21B) $< 5.0\text{B}$ LLM cap and easily pairs with kzap201/phowhisper-large-zaloai (1.61B) for a combined pipeline of 5.815B $< 6.0\text{B}$ total budget.

🚀 Quickstart & Inference

Using with vLLM (Recommended for Fast Serving)

from vllm import LLM, SamplingParams

llm = LLM(
    model="zinzinmit/Qwen3.5-4B-InstructTool",
    max_model_len=16384,
    dtype="bfloat16",
    enable_prefix_caching=True,
    gpu_memory_utilization=0.85,
)

sampling_params = SamplingParams(
    temperature=0.0,
    max_tokens=384,
    stop=["<|im_end|>", "<|endoftext|>"],
)

messages = [
    {"role": "system", "content": "You are LifeMate assistant with tool calling capabilities."},
    {"role": "user", "content": "Tìm cho tôi danh bạ của anh Nam ở công ty."}
]

prompt = llm.get_tokenizer().apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = llm.generate([prompt], sampling_params)
print(outputs[0].outputs[0].text)

Using with Hugging Face Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zinzinmit/Qwen3.5-4B-InstructTool"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

messages = [
    {"role": "system", "content": "You are LifeMate assistant with tool calling capabilities."},
    {"role": "user", "content": "Đặt lịch họp ngày mai lúc 9 giờ sáng."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=384, do_sample=False)

response = tokenizer.decode(output_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

📊 Evaluation Reproduction

To evaluate this model on the LifeMate benchmark:

python scripts/benchmark/run_llm_val.py \
  --model zinzinmit/Qwen3.5-4B-InstructTool \
  --fewshot dynamic \
  --gpu-mem 32