CyberPal-2.0-20B
CyberPal-2.0-20B is a cybersecurity-expert 20B-parameter Small Language Model (SLM) fine-tuned for security operations and threat-management workflows (e.g., CTI Q&A, vulnerability-to-weakness mapping, detection/mitigation recommendations). It is part of the CyberPal 2.0 model family (4B–20B), trained using the SecKnowledge 2.0 data enrichment + formatting pipeline to produce higher-fidelity, task-grounded reasoning traces for cybersecurity tasks. (https://arxiv.org/html/2510.14113v1)
Model Details
- Model type: Decoder-only language model (instruction-tuned)
- Parameters: ~20B
- Base model:
gpt-oss-20b(fine-tuned) - Domain focus: Cybersecurity (CTI, SOC/IR, appsec, IAM, governance/compliance)
- Context length: 8192
- Training data: SecKnowledge 2.0 (see below)
- Paper: "Toward Cybersecurity-Expert Small Language Models" (arXiv:2510.14113)
Intended Use
CyberPal-2.0-20B is intended for defensive cybersecurity use-cases, such as:
- Cyber Threat Intelligence (CTI): answering CTI questions; mapping campaigns/actors/techniques; explaining ATT&CK concepts
- Vulnerability & weakness analysis: correlating CVE evidence / bug tickets → CWE root causes
- Detection & mitigation guidance: proposing detections/mitigations for tactics/techniques/weaknesses/vulnerabilities
- Security operations support: incident summarization, investigation assistance, hypothesis-driven triage, response recommendations
The paper's stated goal is a domain-specialized backbone for "threat management and security operations" that is practical for enterprise/on-prem deployments.
Out-of-scope / not recommended
- Any form of wrongdoing, intrusion, malware development, or instructions intended to enable harm.
- High-stakes decisions without human review (treat outputs as advisory, not authoritative).
- Agentic/Tool use is currently not tested and support will be released in newer versions.
Training Data (SecKnowledge 2.0)
CyberPal-2.0-20B is fine-tuned on SecKnowledge 2.0, produced by an enrichment pipeline that:
- uses expert-in-the-loop schema/format steering (task-specific reasoning formats),
- performs multi-step grounding using documents and/or web search,
- and uses LLM-based judging for readability/factuality checks in the pipeline.
SecKnowledge (the starting dataset) is described as:
- 153k instructions in the first stage (from structured public security sources),
- expanded to a 403k-example cybersecurity corpus in the second stage via synthetic generation.
The SecKnowledge 2.0 pipeline uses gpt-oss-120b (Medium reasoning effort) as the backbone LLM for dataset generation/enrichment.
Note: For full dataset composition/task breakdowns and templates, see the paper's SecKnowledge 2.0 section and appendices.
How to Use
Tips:
- Trained on both short and long reasoning traces, as described in the paper. For harder questions, prompt it to "think step-by-step" before answering.
- vLLM is supported — see the vLLM example below. Note: newer versions of vLLM require setting
export VLLM_USE_FLASHINFER_MOE_FP16=0to avoid numerical issues with MoE layers.
Transformers (example)
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "cyber-pal-security/CyberPal-2.0-20B"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
prompt = """Analyze this C code snippet. Identify the vulnerability, the potential impact, and provide a patched version.
void process_data(char *input) {
char buffer[128];
strcpy(buffer, input);
printf("Processed: %s", buffer);
}"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
temperature=0.0,
)
print(tokenizer.decode(out[0], skip_special_tokens=True))
vLLM (example)
Note: On newer versions of vLLM, you must set the following environment variable before launching the server to avoid numerical issues:
export VLLM_USE_FLASHINFER_MOE_FP16=0
from vllm import LLM, SamplingParams
model_id = "cyber-pal-security/CyberPal-2.0-20B"
llm = LLM(model=model_id, dtype="bfloat16", tensor_parallel_size=1)
prompt = """Analyze this C code snippet. Identify the vulnerability, the potential impact, and provide a patched version.
void process_data(char *input) {
char buffer[128];
strcpy(buffer, input);
printf("Processed: %s", buffer);
}"""
sampling_params = SamplingParams(max_tokens=512, temperature=0.0)
outputs = llm.generate([prompt], sampling_params)
print(outputs[0].outputs[0].text)
Paper
This model is associated with the paper: Toward Cybersecurity-Expert Small Language Models
Citation
If you use this model, please cite:
@misc{levi2025towardcybersecurityexpertsmall,
title={Toward Cybersecurity-Expert Small Language Models},
author={Matan Levi and Daniel Ohayon and Ariel Blobstein and Ravid Sagi and Ian Molloy and Yair Allouche},
year={2025},
eprint={2510.14113},
archivePrefix={arXiv},
primaryClass={cs.CL},
doi={10.48550/arXiv.2510.14113},
url={https://arxiv.org/abs/2510.14113}
}