Qwen3.5-2B · agentic GRPO on SQaLe3

A 2B text-to-SQL agent, not a single-shot SQL generator. Starting from the stock Qwen/Qwen3.5-2B (no supervised warm start), it was trained with GRPO-style RL for 1600 steps against real SQLite execution: it explores a database with tools, then submits a query, and is rewarded on whether the result matches the gold answer.

Training data: SynSQL (simple + moderate). Related models, identical recipe on different training data: cwolff/qwen3.5-2b-grpo-bird (BIRD train) and cwolff/qwen3.5-2b-grpo-sqale (SQaLe3). Both of those score meaningfully higher than this one (~0.55 vs ~0.47 on BIRD full); if you just want the best model of the three, take one of them.

Results — BIRD dev

Measured on this repo as published, loaded straight into vLLM. n=300 BIRD dev rows (seeded), 6-turn agentic episodes, T=0.6 / top-p 0.95 / top-k 20, scored by executing against the real SQLite databases. execution_accuracy_official is BIRD-leaderboard set semantics; strict is order- and duplicate-sensitive. full puts the DDL in the prompt; withheld gives the model only the question and makes it discover the schema through tools.

All five bars were produced by the same harness in the same way, which is what makes them comparable. Two things the chart says out loud:

• RL on a 2B works, and works hardest where the schema is hidden. Against its own starting point the gain is +0.170 with the schema in the prompt and +0.250 with it withheld -- exploration is where the training pays.
• A 13x larger untrained model still wins. Stock Qwen3.8-27B reaches 0.687 / 0.693 without any training on this task. If you have the VRAM for a 27B and do not need a 2B's latency or cost, it is the better model; these checkpoints are for the case where you need the small one.

Episode behaviour: submit rate 0.31 / 0.20 (full/withheld), generated SQL executes 93.7% / 96.0% of the time, 5.3 / 5.8 mean turns.

On the stock-2B column. It is a fresh run of this harness against Qwen/Qwen3.5-2B, not the figure the trainer logged at step 0 (which was 0.13-0.16 on full). The two disagree, and the cause is not settled: the behavioural counters match closely but the fallback-to-last-working-query fires on 7% of episodes here against 41% there, and an untrained model's score is dominated by that fallback. The trained models barely use it (2-7%), which is why their numbers reproduce their training records to within a point or two. The honest reading is that the 0.290 figure is the one measured the same way as every other bar in the chart, and that an untrained baseline in this harness carries more measurement uncertainty than the trained ones do.

Does the published repo match what training measured?

Training could not measure this repo. Its vLLM engine was built from the base model with the checkpoint's weights synced in — a workaround forced by the fact that the raw training export is not loadable by vLLM at all. This repo is the rebuilt, directly-loadable one, so "same model" needed checking rather than assuming.

Both paths were run over the same 300 questions and compared paired (exact McNemar + bootstrap CI), which is the right test here because at n=300 the unpaired noise floor is several points:

full shows no difference. withheld does: the published path scored 5 points HIGHER, and that one clears p<0.05 on its own. Two reasons not to read it as a real gap between the paths. It is one significant result out of six such tests run across these three models, which is roughly what chance alone produces at alpha=0.05 (it does not survive correcting for the six). And this model truncates 35-51% of its episodes, so its run-to-run variance is far larger than its siblings'. The weights in this repo are bit-identical to the training checkpoint's -- that was verified tensor by tensor, not inferred -- so there is no mechanism for the published model to be better. Treat both paths as the same model measured twice.

Two caveats that apply to the absolute numbers regardless:

• The checkpoint was selected on BIRD. Steps 1250-1650 all sit in a 0.41-0.47 band, so treat ~0.45 full as the plateau rather than any single step as a clean result.
• It is the weakest of the three, and that gap is real. Its siblings reach ~0.55 full against this model's 0.46 -- roughly 9 points, which is well outside the measurement floor, unlike the ~1 point separating the other two from each other. See the training-data limitation below for the most likely reason.

Numbers are BIRD dev, produced by this project's own harness, and are not leaderboard submissions.

Serving with vLLM

The repo is a complete Qwen3_5ForConditionalGeneration checkpoint, so it loads directly — no conversion, no --hf-overrides:

vllm serve cwolff/qwen3.5-2b-grpo-synsql --max-model-len 16384

from vllm import LLM, SamplingParams
llm = LLM(model="cwolff/qwen3.5-2b-grpo-synsql", max_model_len=16384)
out = llm.chat(
    [{"role": "system", "content": SYSTEM_PROMPT},
     {"role": "user", "content": "How many singers are there?"}],
    SamplingParams(temperature=0.6, top_p=0.95, top_k=20, max_tokens=2048),
)

generation_config.json stops on both <|im_end|> (248046) and <|endoftext|> (248044), matching the stop set used during RL rollouts. The base model's own config lists only <|endoftext|>, which would run past the end of every assistant turn.

How to prompt it

This model expects an agent loop, and will underperform badly without one. It was trained under the exec_plan protocol: each assistant turn emits a single JSON object and nothing else — no prose, no code fences — and your harness executes it and feeds the result back as the next user turn.

{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}

Tools: list_tables, describe_table, foreign_keys, join_path, sample_rows, run_query, distinct_values, submit_sql. submit_sql ends the episode. Assistant turns open with a <think> block — keep thinking enabled; the chat template does this by default.

The exact system prompt matters. It is EXEC_PLAN_TOOL_INSTRUCTIONS in shared/tools/protocol.py of the SQaLe finetune repo — use that string rather than paraphrasing the tool list above, which is a summary.

Training

Reward is computed by executing the candidate against a real SQLite database built from the row payloads and comparing to the gold result — not by string match against a reference query.

Limitations

• 2B parameters. It is a small model and will lose to a frontier model on hard multi-join questions.
• Trained and evaluated on SQLite only.
• Trained at a 6-turn budget with a 12288-token episode cap. Longer budgets are untested and truncation reached 35-51% at this one.
• Single-turn "schema in, SQL out" prompting is out of distribution — the withheld numbers above are what this model is actually for.
• Inherits the base model's vision tower, which is carried over untouched and was never trained or evaluated here. Treat it as a text model.
• Its training databases were nearly empty. A preflight over the SynSQL subset used here measured a median of 2 rows per table, against 68 for SQaLe3 and 927 for BIRD train. Reward is execution-based, so with two rows per table many different queries return the same trivial result and the signal that separates a correct query from a lucky one is largely absent. This is the most likely cause of the gap to its siblings.
• It burns tool calls. Mean tool calls per episode rose from ~5 to ~50 over training (peaking near 65), with truncation reaching 32-47%, while accuracy stayed flat. Budget for long episodes, or cap turns.

Provenance

Slurm job 26662810, output grpo1800_synsql, checkpoint best_bird/ (step 1600), selected by the trainer on BIRD full official accuracy. Exported by merging the run's 320 trained language-model tensors with the base model's vision tower and MTP head, under the base config.json.