Qwen3.5-2B · agentic GRPO with a length curriculum
A 2B text-to-SQL agent: it explores a database with tools, then submits a query, and was trained with GRPO against real SQLite execution.
- Length curriculum. The sampling pool is reweighted by gold-query length quantile, shifting from short toward long queries over 1000 steps.
- Harder data.
simple+moderate+hard, on a newer SQaLe3 pair. - Double the episode budget — 24,576 tokens against 12,288.
Sibling models, same family:
grpo-bird ·
grpo-synsql
Results
Measured on this repo as published, loaded straight into vLLM, through the
same agentic harness for every row: exec_plan protocol, 6 tool turns,
T=0.6 / top-p 0.95 / top-k 20, scored by executing against real databases.
| benchmark | mode | official | strict | n |
|---|---|---|---|---|
| BIRD dev | schema in prompt | 0.537 | 0.497 | 300 |
| BIRD dev | schema withheld | 0.473 | 0.433 | 300 |
| BIRD dev, challenging only | schema in prompt | 0.347 | 0.313 | 144 |
| BIRD dev, challenging only | schema withheld | 0.326 | 0.292 | 144 |
| SQaLe3 test | withheld, 64-table cap | 0.573 | 0.540 | 300 |
| SQaLe3 test | withheld, full schema | 0.583 | 0.533 | 300 |
| Spider 2.0-lite | schema in prompt | 0.046 | — | 87 |
| Spider 2.0-lite | schema withheld | 0.046 | — | 87 |
BIRD and SQaLe3 use BIRD-official set semantics. Spider 2.0 uses its own upstream comparator on its local-SQLite subset (87 of 547 questions), is not numerically comparable to the other two, and is not a leaderboard number.
What the RL bought, on the hardest questions
BIRD tags 145 of its 1,534 dev questions challenging. On all of them, same
harness, paired on the question:
| model | params | schema shown | withheld |
|---|---|---|---|
| Qwen3.5-2B (untrained) | 2B | 0.083 | 0.132 |
| this model | 2B | 0.347 | 0.326 |
| Qwen3.5-9B (untrained) | 9B | 0.431 | 0.333 |
| Qwen3.8-27B (untrained) | 27B | 0.604 | 0.583 |
Against its own untrained base the gain is large and unambiguous — paired
McNemar p < 0.0001 in both modes. Against an untrained 9B it is behind with
the schema shown, but level when the schema is withheld (0.326 vs 0.333,
p = 0.74). That is the regime it was trained in (EXPLORE_SHARE=1.0), and it is
where a 2B trades evenly with a model 4.5× its size. The 9B's submit rate
collapses from 0.34 to 0.06 when it has to discover the schema itself; this
model's holds.
Does the published repo match what training measured?
Training builds its vLLM engine from the base model and syncs the checkpoint's weights in; this repo is the rebuilt, directly-loadable one. Compared paired over the same 300 questions:
| mode | weight-sync path | this repo | paired delta | McNemar p |
|---|---|---|---|---|
| schema shown | 0.553 | 0.537 | −0.017, 95% CI [−0.050, +0.020] | 0.46 |
| withheld | 0.523 | 0.473 | −0.050, 95% CI [−0.090, −0.010] | 0.020 |
The withheld row is significant, so state it plainly: on that mode the
directly-loaded repo scored 5 points below the weight-sync path. The
weights are bit-identical to the training checkpoint — verified tensor by
tensor, not inferred — so there is no mechanism for the two to differ, and this
is one of two significant results across ten such paired tests on this model
family (the other went the opposite way). The most likely explanation is
run-to-run sampling variance under continuous batching, which is large for a
model that truncates a third of its episodes. It is reported rather than
buried; if it matters to you, re-run the eval at a different seed.
Serving
vllm serve cwolff/qwen3.5-2b-grpo-curriculum --max-model-len 24576
generation_config.json stops on both <|im_end|> (248046) and <|endoftext|>
(248044), matching the stop set used during RL rollouts. Give it a larger
context than its siblings — it was trained at a 24,576-token episode budget
and uses it.
How to prompt it
It expects an agent loop and will underperform badly without one. Each assistant turn emits a single JSON object and nothing else; your harness executes it and feeds the result back as the next user turn.
{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
Tools: list_tables, describe_table, foreign_keys, join_path,
sample_rows, run_query, distinct_values, submit_sql. submit_sql ends
the episode. Assistant turns open with a <think> block — keep thinking on.
The exact system prompt is EXEC_PLAN_TOOL_INSTRUCTIONS in
shared/tools/protocol.py; use that string rather than paraphrasing.