Qwen2.5-Coder-7B BIRD CoT (Text-to-SQL)

A 7B text-to-SQL model that reasons step by step over a database schema and then writes the SQL. It is fine-tuned from Qwen/Qwen2.5-Coder-7B-Instruct by distilling execution-verified chain-of-thought solutions.

On BIRD dev (realistic, messy schemas, the harder text-to-SQL benchmark) it reaches 52.1% greedy result accuracy and 58.5% with self-consistency (Best-of-N, K=8). With self-consistency it matches DeepSeek V4-Pro (1.6T parameters) at roughly 0.4% of the size, running locally.

Results: BIRD dev (execution result accuracy)

All models use a single chain-of-thought sample unless noted, so the comparison is like for like.

Result accuracy is the fraction of queries whose SQL executes to the same rows as the gold query (BIRD's official execution metric). The larger frontier models score higher, as expected. The point is that a 7B reaches their accuracy band at a fraction of the size and cost. The 58.5% figure uses K=8 self-consistency (roughly 8x inference).

Usage

Prompt the model to reason step by step. It returns the reasoning followed by a fenced SQL block. Take the last SQL block as the query.

import re, torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jk200201/qwen2.5-coder-7b-bird-cot"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

SYSTEM = ("You are an expert SQLite query writer. Reason step by step about the schema "
          "and the question, then output the final query in a fenced sql code block.")

def build_prompt(schema, question):
    return ("Given the database schema and question, work out the correct SQLite query step by step.\n\n"
            f"Database Schema:\n{schema}\n\nQuestion: {question}\n\n"
            "Think step by step, then give the final answer in a fenced sql block.")

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": build_prompt(
        "CREATE TABLE singer (Singer_ID INT, Name TEXT, Age INT);",
        "How many singers are older than 40?")},
]
text   = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out    = model.generate(**inputs, max_new_tokens=512, do_sample=False)
resp   = tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
sql = re.findall(r"```(?:sql)?\s*(.*?)```", resp, re.DOTALL)[-1].strip()
print(sql)

For best accuracy (58.5%), sample K=8 at temperature 0.8, execute each candidate, and take the majority result (self-consistency).

Model family

Training

Reasoning distillation (CoT-SFT). A strong teacher (Qwen3-Coder-480B) generated step-by-step CoT solutions on BIRD train. Only execution-verified-correct chains were kept (5,593 examples), then supervised fine-tuned into the 7B. Distilling the teacher's reasoning generalized across BIRD's cross-domain dev databases better than distilling SQL answers directly.

Config: QLoRA (4-bit NF4, LoRA rank 32, alpha 64), 2 epochs, learning rate 2e-4 cosine, max sequence length 8192.

Limitations

• Tuned for BIRD-style analytic SQL over realistic schemas. Unusual dialects or domains may need adaptation. It emits SQLite dialect.
• Greedy (52.1%) is the deployable single-shot number. The 58.5% figure needs K=8 self-consistency (roughly 8x inference).
• This is a 7B model. Review generated SQL before running it on production data.

Citation

@misc{qwen25coder7b_bird_cot_2026,
  title  = {Qwen2.5-Coder-7B BIRD CoT: reasoning distillation for text-to-SQL},
  author = {Jenish Kothari},
  year   = {2026}
}

Acknowledgements

Base model: Qwen2.5-Coder by Alibaba Qwen. Teacher: Qwen3-Coder-480B. Benchmark: BIRD (bird-bench.github.io).