Cyber-Prime 1 (2.6B)
A fine-tuned cybersecurity specialist built on LiquidAI/LFM2-2.6B. Despite having only 2.6 billion parameters, Cyber-Prime 1 outperforms Llama-2-7B on every CyberBench task and beats GPT-3.5-Turbo on named entity recognition and threat intelligence summarization.
Overview
Cyber-Prime 1 is a surgical fine-tune of the LFM2.5-2.6B base model, trained on a curated mix of:
- NER repair data — 6,000+ rows fixing JSON format extraction for cybersecurity entities
- HTTP reasoning traces — 5,000 rows with authored chain-of-thought reasoning for attack detection (XSS, SQLi, path traversal, command injection)
- Email classification — 5,000 direct-mode rows for phishing vs. safe classification
- CyNews summarization — 2,000 rows for threat intelligence headline generation
- Source data — 2,000 rows from GHSA, KEV, and ATT&CK sources
- Multiple choice — security knowledge and cyber quiz gold rows
The model uses two distinct modes:
- Direct mode for simple classification (email phishing, HTTP detection) and summarization
- Think mode (chain-of-thought) for tasks benefiting from reasoning (HTTP analysis, NER extraction)
Benchmark Results
Evaluated on CyberBench (Liu et al., AAAI-24 AICS Workshop).
| Dataset | Metric | GPT-4 | GPT-3.5 Turbo | Mistral-7B Instruct | Llama-2-7B | Cyber-Prime 1 (2.6B) |
|---|---|---|---|---|---|---|
| CyNER | F1 | 0.554 | 0.334 | 0.323 | 0.263 | 0.382 |
| APTNER | F1 | 0.500 | 0.409 | 0.262 | 0.280 | 0.413 |
| CyNews | ROUGE-1 | 0.275 | 0.271 | 0.217 | 0.003 | 0.354 |
| SecMMLU | Accuracy | 0.830 | 0.780 | 0.720 | 0.630 | 0.580 |
| CyQuiz | Accuracy | 0.810 | 0.830 | 0.690 | 0.620 | 0.570 |
| F1 | 0.939 | 0.789 | 0.889 | 0.942 | 0.728 | |
| HTTP | F1 | 0.841 | 0.831 | 0.472 | 0.428 | 0.483 |
| Average | — | 0.721 | 0.609 | 0.511 | 0.451 | 0.501 |
Key Results
- Beats Llama-2-7B on all 7 tasks — a 2.6B model sweeping a 7B model across the board
- Beats Mistral-7B-Instruct on 4/7 tasks — APTNER (+0.151), CyNews (+0.137), CyNER (+0.059), HTTP (+0.011)
- Beats GPT-3.5-Turbo on 3/7 tasks — CyNews (+0.083), APTNER (+0.004), CyNER (+0.048)
- Beats GPT-4 on CyNews — 0.354 vs 0.275 ROUGE-1 (+0.079), a 2.6B model out-summarizing GPT-4
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Akahsizrr/Cyber-Prime-1-2.6B",
torch_dtype="auto",
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Akahsizrr/Cyber-Prime-1-2.6B")
# NER extraction (Alpaca format)
prompt = """### Instruction:
Extract cybersecurity entities from the given text.
### Input:
APT29 used WELLMAIL to compromise Microsoft Exchange servers via CVE-2021-26855.
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Limitations
- Knowledge tasks (SecMMLU, CyQuiz): Limited by parameter count — a 2.6B model cannot store broad cybersecurity knowledge as well as larger models
- Email classification: Slightly below GPT-3.5-Turbo due to reasoning mode interference from HTTP training data
- HTTP detection: Reasoning improves detection of obvious attacks but may miss subtle injection patterns
- Not a security tool: This model is a research artifact for benchmark evaluation, not a production security system
Citation
@misc{cyberprime1,
title={Cyber-Prime 1: A Small Cybersecurity Language Model},
author={Akahsizrr},
year={2025},
url={https://huggingface.co/Akahsizrr/Cyber-Prime-1-2.6B}
}
@misc{liu2024cyberbench,
title={Cyberbench: A multi-task benchmark for evaluating large language models in cybersecurity},
author={Liu, Zefang and Shi, Jialei and Buford, John F},
howpublished={AAAI-24 Workshop on Artificial Intelligence for Cyber Security (AICS)},
year={2024}
}