Qwen3.5-DeltaCoder-9B

Reliable tool-calling for agentic coding — LoRA fine-tune of Qwen3.5-9B v1.1-DPO released — DPO alignment improves code correctness and self-verification. If you downloaded before March 28, 2026, please re-pull to get v1.1-DPO.

Small language models can reason about code, but they struggle to call tools reliably. DeltaCoder takes a strong reasoning base and teaches it to produce correctly-formatted JSON tool calls — the kind that coding agents like OpenCode, Pi, and Cline depend on.

v1.1-DPO adds Direct Preference Optimization to further improve code correctness — the model now self-corrects its own bugs rather than submitting wrong answers.

Downloads

The Problem

Jackrong's Qwen3.5-9B reasoning distill scores 53.7% on HumanEval — best-in-class at 9B. But when used as a coding agent, it frequently produces malformed JSON tool calls:

tool=edit, error=JSON Parse error: Property name must be a string literal
tool=bash, error=JSON Parse error: Expected '}'

DeltaCoder fixes this, and v1.1-DPO further improves code correctness through preference learning.

What's New in v1.1-DPO

• Self-correcting behavior — detects and fixes its own bugs during agentic tasks
• Improved code correctness — trained on 4,519 preference pairs from AceCode-V2-122K
• Two-stage merge — v1 SFT tool-calling improvements + DPO code quality improvements combined
• 13 GGUF quants — from Q2_K to BF16, covering all VRAM configurations

Training Details

v1 — SFT (Tool-Call Reliability)

v1.1 — DPO (Code Correctness)

LoRA Target Modules

All major weight matrices adapted across the hybrid architecture:

• Full Attention (8/32 layers): q_proj, k_proj, v_proj, o_proj
• Gated Delta Net (24/32 layers): in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj
• MLP (all 32 layers): gate_proj, up_proj, down_proj

Usage

Ollama

ollama create deltacoder -f Modelfile

llama.cpp / ik_llama.cpp

./llama-server -m DeltaCoder-9B-v1.1-DPO-Q5_K_M.gguf -ngl 999 -c 131072 -ctk f16 -ctv q4_0 -fa 1 --jinja

With PEFT (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "danielcherubini/Qwen3.5-DeltaCoder-9B")
tokenizer = AutoTokenizer.from_pretrained("danielcherubini/Qwen3.5-DeltaCoder-9B")

Benchmarks

*v1.1-DPO timed out on 2 tasks that v1 answered incorrectly — behavioral improvement confirmed, re-evaluating with extended timeout.

Recommended Sampling Settings

[!WARNING] Do not use temperature below 0.5 — low temperatures cause deterministic looping in multi-turn agentic use.

KV Cache Quantization

Key Findings

[!NOTE] Qwen3.5 is a VLM — Unsloth treats it as a vision model. For text-only DPO training, use standard HuggingFace + PEFT + TRL directly (no Unsloth DPOTrainer).

[!WARNING] Do not use flash_attention_2 with sample packing on Qwen3.5 — training loss goes to 0. Use attn_implementation="eager" instead.

• Qwen3.5 uses Gated Delta Networks — include in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj in LoRA target modules or 75% of attention layers are untrained
• DPO pairs generated on-policy using Qwen/Qwen3.5-9B base with vLLM async inference (32 concurrent requests)
• Keep rate of 45.2% from 10K AceCode problems (4,519 pairs used for training)

Project Structure

scripts/
  train_unsloth.py          # v1 SFT training
  train_dpo.py              # v1.1 DPO training (HF + PEFT + TRL)
  generate_dpo_pairs.py     # Async on-policy pair generation
  merge_and_export_dpo.py   # Two-stage merge + GGUF export

Status

• v1 SFT fine-tune (CoderForge, H200, ~10hrs)
• GGUF export (all quants Q2_K → BF16)
• HumanEval benchmarking (50.6% / 49.4%)
• Terminal-Bench evaluation (2/4 easy tasks)
• DPO pair generation (4,519 pairs from AceCode-V2-122K)
• v1.1-DPO training (H100, ~3.7hrs)
• v1.1-DPO GGUF export + HuggingFace release
• v1.1-DPO HumanEval benchmarking
• v1.1-DPO Terminal-Bench extended timeout evaluation

Acknowledgements

• Unsloth for Qwen3.5 SFT training support
• Together AI for the CoderForge dataset
• TIGER Lab for AceCode-V2-122K
• Jackrong for the reasoning distillation
• Qwen for the base model