KernelZero-CUDA-SFT-7B
KernelZero-CUDA-SFT-7B is the supervised CUDA kernel generation checkpoint used to initialize the CUDA Coder in KernelZero. It is based on Qwen2.5-Coder-7B-Instruct and trained on 42,454 CUDA code-generation examples.
Paper
KernelZero: Continual Co-Evolution of LLMs for Kernel Generation
KernelBench results
The paper reports the following results with 10 sampled candidates per problem.
| Benchmark | pass@1 | pass@5 | pass@10 | fast_1@1 | fast_1@10 | fast_2@1 | fast_2@10 |
|---|---|---|---|---|---|---|---|
| Level 1 | 70.6 | 94.77 | 98.0 | 15.1 | 29.0 | 6.3 | 14.0 |
| Level 2 | 59.0 | 90.33 | 94.0 | 2.0 | 9.0 | 1.3 | 6.0 |
Intended use
This checkpoint generates CUDA extensions from PyTorch modules. Use the KernelZero CUDA prompt and validation environment for reproducible evaluation.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "kcxain/KernelZero-CUDA-SFT-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
Checkpoint provenance
This release contains the merged Hugging Face inference weights from SFT checkpoint 3606. Optimizer states, scheduler state, RNG state, and distributed training artifacts are excluded.