Qwen3.5-9B-metro-v24

QLoRA fine-tune of Qwen3.5-9B for the MetroLLM-Bench transit-kiosk task: multi-step tool calling, structured terminal-state synthesis, and disruption/advisory handling across six metro systems.

v24 is the leakage-free retraining used in the MetroLLM-Bench paper. Teacher distillation traces are drawn only from the 717-case training partition of a stratified 75/25 split (seed=42); the 238-case held-out partition is strictly reserved for reporting. It supersedes continker/Qwen3.5-9B-metro-v23, which was trained on the full case set.

Held-out results (n=238, mean of 2 training seeds)

Contents

• adapter/ — LoRA adapter (rank 16, α 32; QLoRA 4-bit NF4 training) with tokenizer + chat template
• Qwen3.5-9B-metro-v24-Q4_K_M.gguf — merged + quantised GGUF (5.3 GB), self-contained
• training_summary.json — training metadata

Usage

GGUF (recommended, self-contained):

llama-server --hf-repo continker/Qwen3.5-9B-metro-v24 --hf-file Qwen3.5-9B-metro-v24-Q4_K_M.gguf

The LoRA adapter is provided for merging. Its keys use the .language_model. module prefix; to apply it to a text-only Qwen3.5-9B, strip that prefix from the adapter keys first (see the MetroLLM-Bench demo _localise_adapter).

Training

QLoRA, rank 16, 3 epochs, single RTX 5090. Distilled from Qwen3.5-27B-dense and 35B-A3B teachers (tier-1 ≥ 90% filter, 600 examples). Recipe and full evaluation: MetroLLM-Bench. License Apache 2.0 (inherits Qwen3.5-9B).

Links

• Paper (arXiv:2609.10016)
• Paper page (Hugging Face)
• Technical report v1.2 (PDF)
• Benchmark and code
• Live demo
• All four v24 students (collection)

Citation

@techreport{hendriks2026metrollm,
  title       = {MetroLLM-Bench: Evaluating Language Models as Transit Kiosk Runtimes},
  author      = {Hendriks, Remco},
  institution = {Continker},
  type        = {Technical report},
  number      = {v1.2},
  year        = {2026},
  month       = {9},
  doi         = {10.5281/zenodo.21893944},
  eprint      = {2609.10016},
  archiveprefix = {arXiv},
  primaryclass = {cs.LG},
  url         = {https://arxiv.org/abs/2609.10016}
}