qwen3.5-0.8b-astcss-t5
English → astcss selector (sitting_duck's ast_select) at 0.8B, trained on the tier-5 corpus.
This is the successor to qwen3.5-0.8b-astcss.
Same model size, same card, same recipe — only the training corpus changed (820 pairs / 2,205
rows → 6,053 pairs / 15,928 rows / 2,599 distinct selector shapes).
Scores
Measured on this merged artifact, not carried over from the adapter — an fp16 merge is verified, not assumed. Both numbers reproduce the adapter tier-for-tier.
| eval | this model | predecessor (820-pair corpus) |
|---|---|---|
| 108-pair selector eval | 82.4 % (89/108) | 82.4 % (89/108) |
eval_t5 — 55 harder, relational selectors | 83.6 % (46/55) | 34.5 % |
Scoring is by execution: the selector is run against the fixture and the returned node set is
compared to the reference. A different string selecting the same nodes counts as correct — which
matters, since exact-string agreement is only 56.4 % on eval_t5.
For scale on that second eval: a 4 B model trained on the old corpus scores 67.3 %, and an untuned 9 B scores 60.0 %. Tier-5 training is worth more here than either.
The 108-pair eval is saturated at this model size — the old corpus, this corpus at epoch 1 and
at epoch 2 all land on exactly 89/108. Only the tier distribution moves. Read eval_t5 for the
difference between these two models.
It needs the card
This model scores 0.0 % without its system prompt, at every model size we have tested. It was
trained with a per-language vocabulary card in the system position; without one it answers as a
chat model rather than emitting a selector. Cards live in the
astcss-eval repo under train/cards/card_<lang>.md.
Greedy decoding, thinking off, ~6 completion tokens per answer, ~0.24 s median latency.
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
m = "teaguesterling/qwen3.5-0.8b-astcss-t5"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.float16, device_map="cuda")
card = open("train/cards/card_python.md").read() # REQUIRED
msgs = [{"role": "system", "content": card},
{"role": "user", "content": "functions that call range without any try block"}]
ids = tok(tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False),
return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=48, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
# .fn:has(.call#range):not(:has(.try))
What it is
LoRA (r=16, α=32, all-linear) on Qwen3.5-0.8B, fp16, 1 epoch, seed 18, per-language card, template cap 8. Merged into the base weights — no adapter needed at serving time. The merge was verified by tensor delta: 186 targeted tensors at max |Δ| 0.036 against 134 untouched at 0.0005, which is fp16 round-trip noise.
Epoch 1, deliberately. Epoch 2 had the better validation loss (0.0557 vs 0.0833) and the worse
eval (80.0 % vs 83.6 % on eval_t5, identical on the 108-pair). Validation loss does not track the
eval here.
Limitations
- Text-only.
save_pretrainedunderAutoModelForCausalLMdrops the base's vision tower (153model.visual.*) and multi-token-prediction head (15mtp.*) — 168 base tensors are not written. If you need the multimodal stack, merge differently. - Needs the card, as above. This is not a general chat model.
- Trained and evaluated on Python, Bash, C, C++, Go, Java, JavaScript, Rust and SQL fixtures.
eval_t5is 55 pairs; a 2-pair difference there is within noise.