Winnow pooled line classifier (32M)
The default model of Winnow, a task-aware pruner of coding-agent tool output for Headroom.
Given a task query and one tool output (a test run, build log, grep result, diff...), it scores every line by how much the agent needs it for its next step. Winnow keeps the high-scoring lines verbatim, always keeps errors and tracebacks, and replaces the rest with retrievable markers.
Results
On the 618-example test split of
KRLabsOrg/tool-output-extraction-swebench,
inside Winnow (consumer GPU, float32):
| Winnow + this model | Headroom relevance_split (hybrid) | |
|---|---|---|
| Gold-line recall | 0.847 | 0.753 |
| Token reduction | 41.4% | 57.6%¹ |
| Error lines lost | 0 | 7,771 |
| Latency p50 / p95 | 0.29 s / 0.92 s | 2.2 s / 8.6 s |
At equal compression (50 / 70 / 90% of lines dropped) it keeps 0.89 / 0.84 /
0.68 of the gold lines, against 0.73 / 0.63 / 0.46 for relevance_split.
¹ Upper bound: inside Headroom the dropped tail is Kompressed, not removed.
Use with Winnow
pip install "headroom-winnow[model]"
headroom proxy --compressor winnow
Winnow downloads this model on first use.
Use directly
The model ships its own code, so load it with trust_remote_code:
from transformers import AutoModel, AutoTokenizer
repo = "rbk4209/winnow-pooled-32m"
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo)
result = model.process(
task="Find the traceback that shows the import error",
tool_output=open("pytest.log").read(),
tokenizer=tokenizer,
threshold=0.5,
return_line_probabilities=True,
)
print(result["highlighted_lines"])
Training
- Base model:
jhu-clsp/ettin-encoder-32m(MIT) - Data: 10,508 training examples of
KRLabsOrg/tool-output-extraction-swebench(Apache-2.0) - Code: Squeez's pooled line classifier (
squeez.encoder.train --classifier-type pooled), at a pinned commit - Recipe: 4,096-token windows, effective batch 96, learning rate 5e-5, cosine schedule, 3 epochs, fp16, one T4 (~6 h). Best validation loss 0.31.
The full notebook is in the Winnow repository:
training/kaggle_train_pooled.ipynb.
Limitations
- English tool output from software-engineering tasks; other domains are untested.
- It ranks lines; it does not summarise or rewrite. Use it with Winnow's safety rules (or your own) rather than trusting it to keep every error line alone: the bare model drops failure lines that Winnow's rules put back.
Credits
Built on the Squeez approach, training code and dataset by KRLabs and the Ettin encoder by JHU CLSP. Licensed under Apache-2.0.