qwen3.5-4b-astcss
English → astcss selector (sitting_duck's ast_select) at 4B. The strongest model we have on
the base selector eval.
Scores
Measured on this merged artifact, scored in NF4 to match how the adapter was measured. A merge is verified, not assumed — and this one is not identity:
| eval | this model | the adapter it came from |
|---|---|---|
| 108-pair selector eval | 88.9 % (96/108) | 89.8 % (97/108) |
eval_t5 — 55 harder, relational selectors | 65.5 % (36/55) | 67.3 % (37/55) |
The merge cost exactly one pair on each eval. That is within noise, but it is real: this adapter was
fitted against NF4 weights, and folding it into fp16 is a lossy operation. verify_merge shows
why the check matters — 248 targeted tensors at max |Δ| 0.0031 against 178 untouched at 0.00049.
The targeted deltas are only ~6× the fp16 round-trip floor, a far narrower margin than an
fp16-trained adapter gives.
Tier breakdown on the 108-pair eval: 21/21, 20/25, 28/31, 27/31.
Scoring is by execution — the selector runs against the fixture and the returned node set is compared to the reference, so a different string selecting the same nodes counts as correct.
Read this before assuming bigger is better
On the harder eval_t5, a 0.8B model
(qwen3.5-0.8b-astcss-t5) scores
83.6 % against this model's 65.5 %. That 0.8B is five times smaller and wins by 18 points,
because it was trained on a 7.4× larger corpus. This 4B was trained on the older, smaller one.
Corpus beat parameter count, decisively, on the eval that discriminates. Use this model for the basic selector eval; use the 0.8B for relational and compound selectors.
It needs the card
0.0 % without its system prompt, at every model size tested. Trained with a per-language
vocabulary card in the system position. Cards live in the
astcss-eval repo under train/cards/card_<lang>.md.
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
m = "teaguesterling/qwen3.5-4b-astcss"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.float16, device_map="cuda")
card = open("train/cards/card_python.md").read() # REQUIRED
msgs = [{"role": "system", "content": card},
{"role": "user", "content": "functions that call range without any try block"}]
ids = tok(tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False),
return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=48, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
# .fn:has(.call#range):not(:has(.try))
Greedy decoding, thinking off, ~5 completion tokens per answer.
What it is
QLoRA (r=16, α=32, all-linear) on Qwen3.5-4B in NF4, 2 epochs, per-language card, template cap 8. Merged into the base weights — no adapter at serving time.
Limitations
- Text-only. 312 base tensors are not written: the vision tower (
model.visual.*) and the multi-token-prediction head (mtp.*) are dropped bysave_pretrainedunderAutoModelForCausalLM. - Needs the card. Not a general chat model.
- NF4-fitted, fp16-merged. Scored here in NF4; fp16 inference may differ slightly.
- Beaten on relational selectors by a model five times smaller — see above.