kernel-coder-1.5b
A 1.5B code model that writes C in Linux kernel style — tab indentation,
brace placement, declarations before statements, -ERRNO returns, goto label
unwinding — while keeping the base model's Python ability.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "nethunter2023/kernel-coder-1.5b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="bfloat16", device_map="auto")
messages = [
{"role": "system", "content": "You are a Linux kernel developer. Reply with a "
"single C code block containing only the function."},
{"role": "user", "content": "Implement `int demo_probe(struct device *dev)`: "
"allocate a private struct and unwind on error."},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids.to(model.device), max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
Chat template ships with the tokenizer. Greedy decoding; the answer is the last fenced code block.
Results — kernel C style
N = 40 held-out kernel-doc tasks, greedy decoding, scored with the kernel's
own scripts/checkpatch.pl --no-tree --file --strict, reported as weighted
defects per line: (2*errors + warnings + 0.5*checks) / lines. All models were
given the identical prompts and scored by identical code.
| params | defects / line ↓ | 95% CI | checkpatch errors ↓ | idiom ↑ | |
|---|---|---|---|---|---|
| deepseek-coder-1.3b-instruct | 1.3B | 0.679 | ±0.196 | 4.43 | 0.738 |
| Qwen2.5-Coder-3B-Instruct | 3B | 0.750 | ±0.182 | 4.78 | 0.755 |
| Qwen2.5-Coder-1.5B-Instruct (base) | 1.5B | 0.872 | ±0.185 | 5.58 | 0.664 |
| kernel-coder-1.5b | 1.5B | 0.020 | ±0.012 | 0.00 | 0.995 |
| the kernel's own code | — | 0.017 | — | 0.00 | 1.000 |
Against Qwen2.5-Coder-3B-Instruct — twice the parameters — the paired
difference is −0.73 defects/line, 95% CI [−0.91, −0.55], t = −7.87, lower on
33 of 40 tasks. No general-purpose code model tested comes close, and this
model sits within noise of the kernel's own source.
Results — Python
MBPP test, 200 problems, greedy, executing the dataset's assertions.
| params | pass@1 | |
|---|---|---|
| deepseek-coder-1.3b-instruct | 1.3B | 0.250 |
| kernel-coder-1.5b | 1.5B | 0.420 |
| Qwen2.5-Coder-1.5B-Instruct (base) | 1.5B | 0.420 |
| Qwen2.5-Coder-3B-Instruct | 3B | 0.535 |
Python is unchanged from base — the kernel specialisation cost nothing, and gained nothing, here. A 3B model is still better at general Python.
If you re-run MBPP, strip the trailing print(...) / assert / __main__
statements the model appends after the function before executing. They run at
import time and abort otherwise-correct solutions; leaving them in costs roughly
3 points.
Limitations
- The kernel gains are stylistic and structural, not functional. Kernel code cannot be executed in a sandbox, so nothing here measures semantic correctness. A well-formatted stub and a working implementation score alike. Review output before use.
- A large share of the checkpatch improvement is indentation. The base model indents kernel C with spaces; this one uses tabs, and checkpatch flags every space-indented line.
- Roughly half of kernel completions leave part of the body as placeholder comments rather than a full implementation — a rate unchanged from base.
- N = 40 on the kernel evaluation. The margin over the baselines is large relative to that, but finer distinctions would need a bigger set.
- Training methodology is not published.
Base model: Qwen/Qwen2.5-Coder-1.5B-Instruct