Qwen3.5-2B · agentic GRPO with a length curriculum

A 2B text-to-SQL agent: it explores a database with tools, then submits a query, and was trained with GRPO against real SQLite execution.

It differs from its three siblings in four ways that matter:

• Warm-started, not trained from the stock base — it continues cwolff/qwen3.5-2b-grpo-sqale from its step-1600 checkpoint.
• Length curriculum. The sampling pool is reweighted by gold-query length quantile, shifting from short toward long queries over 400 steps.
• Harder data. simple + moderate + hard, on a newer SQaLe3 pair.
• Double the episode budget — 24,576 tokens against 12,288.

Sibling models, same family, no curriculum: grpo-bird · grpo-sqale · grpo-synsql

This repo serves the run's LAST checkpoint (step 600). It previously served step 300, which the trainer had selected on BIRD. Measured on a metric neither was selected for, the two are indistinguishable, and step 600 answers with less than half the tool calls — see Which checkpoint below.

Results

Measured on this repo as published, loaded straight into vLLM, through the same agentic harness for every row: exec_plan protocol, 6 tool turns, T=0.6 / top-p 0.95 / top-k 20, scored by executing against real databases.

BIRD and SQaLe3 use BIRD-official set semantics. Spider 2.0 uses its own upstream comparator on its local-SQLite subset (87 of 547 questions), is not numerically comparable to the other two, and is not a leaderboard number.

What the RL bought, on the hardest questions

BIRD tags 145 of its 1,534 dev questions challenging. On all of them, same harness, paired on the question:

Against its own untrained base the gain is large and unambiguous — paired McNemar p < 0.0001 in both modes. Against an untrained 9B it is behind with the schema shown, but level when the schema is withheld (0.326 vs 0.333, p = 0.74). That is the regime it was trained in (EXPLORE_SHARE=1.0), and it is where a 2B trades evenly with a model 4.5× its size. The 9B's submit rate collapses from 0.34 to 0.06 when it has to discover the schema itself; this model's holds.

Which checkpoint

The run saved a BIRD-selected checkpoint at step 300 and ran to step 600. Both were measured on everything:

Each checkpoint wins the metric it was chosen on — symmetric selection bias, and a reason to trust neither number as a comparison. On the neutral metric the paired difference is +0.028 (95% CI [−0.034, +0.090], p = 0.52): no difference. Step 600 ships because it is equal there and roughly half the inference cost. If you want the higher BIRD-overall number, take step 300 from the run directory; it is not better, it is differently selected.

Does the published repo match what training measured?

Training builds its vLLM engine from the base model and syncs the checkpoint's weights in; this repo is the rebuilt, directly-loadable one. Compared paired over the same 300 questions:

The withheld row is significant, so state it plainly: on that mode the directly-loaded repo scored 5 points below the weight-sync path. The weights are bit-identical to the training checkpoint — verified tensor by tensor, not inferred — so there is no mechanism for the two to differ, and this is one of two significant results across ten such paired tests on this model family (the other went the opposite way). The most likely explanation is run-to-run sampling variance under continuous batching, which is large for a model that truncates a third of its episodes. It is reported rather than buried; if it matters to you, re-run the eval at a different seed.

Honest caveats

• The run did not finish. 1,000 steps were planned; it was cancelled at ~step 656 after 1d19h. This is step 600.
• It still over-explores. 15 mean tool calls with the schema shown and 29 withheld — half what step 300 used, but well above the ~13 of its siblings — and 17%/11% of BIRD episodes hit the token cap. On Spider 2.0 truncation is 78%, so most of those episodes never finish.
• Spider 2.0 is near zero (0.046 = 4 of 87), inside the noise floor at this n. An untrained Qwen3.8-27B scores 0.414 on the same subset.
• BIRD withheld is below its grpo-bird sibling (0.473 vs 0.547).

Serving

vllm serve cwolff/qwen3.5-2b-grpo-curriculum --max-model-len 24576

generation_config.json stops on both <|im_end|> (248046) and <|endoftext|> (248044), matching the stop set used during RL rollouts. Give it a larger context than its siblings — it was trained at a 24,576-token episode budget and uses it.

How to prompt it

It expects an agent loop and will underperform badly without one. Each assistant turn emits a single JSON object and nothing else; your harness executes it and feeds the result back as the next user turn.

{"tool": "list_tables"}
{"tool": "describe_table", "args": {"table_name": "singer"}}
{"tool": "run_query", "args": {"sql": "SELECT COUNT(*) FROM singer"}}
{"tool": "submit_sql", "args": {"sql": "SELECT COUNT(*) FROM singer"}}

Tools: list_tables, describe_table, foreign_keys, join_path, sample_rows, run_query, distinct_values, submit_sql. submit_sql ends the episode. Assistant turns open with a <think> block — keep thinking on. The exact system prompt is EXEC_PLAN_TOOL_INSTRUCTIONS in shared/tools/protocol.py; use that string rather than paraphrasing.

Training

Reward executes the candidate against a real SQLite database built from the row payloads and compares to the gold result — not string match against a reference query.

Limitations

• 2B parameters; it will lose to a frontier model on hard multi-join questions.
• Trained and evaluated on SQLite only.
• Single-turn "schema in, SQL out" prompting is out of distribution.
• Inherits the base model's vision tower, carried over untouched and never trained or evaluated here. Treat it as a text model.
• Provenance: Slurm job 26862235, output grpo_curriculum_len, checkpoint checkpoint-step-600 (byte-identical to the run's best/). Exported by merging the run's 320 trained language-model tensors with the base model's vision tower and MTP head under the base config.json.