Qwen2.5-3B-CyberSec-Instruct
A fine-tuned version of Qwen2.5-3B-Instruct specifically designed for advanced cybersecurity analysis. This model is built to bridge the gap between high-level vulnerability descriptions and low-level exploit code execution.
- Developed by: Mohamedabul
- License: apache-2.0
- Finetuned from model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
- Architecture: 3B parameters (4-bit QLoRA)
This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.
Model Capabilities
This LLM acts as an expert cybersecurity analyst and reverse engineer. It is capable of:
- Vulnerability Triage: Automatically generating structured severity, attack vector, and mitigation reports for any CVE.
- Exploit Reverse-Engineering: Analyzing raw exploit code (C, Python, Bash) to provide an immediate technical breakdown of how the exploit works and what vulnerabilities it targets.
- Attack Chain Reasoning: Combining a CVE with raw exploit code to generate a step-by-step kill-chain analysis, from initial access to system compromise.
Training Data
To achieve maximum accuracy, the model was fine-tuned on an expansive historical corpus of modern vulnerabilities and exploits, completely uncapped and unfiltered:
- NVD CVE Database: Vulnerabilities published between 2020 through 2025.
- Exploit-DB: Over 45,000+ real-world exploits directly from Offensive Security.
- MITRE CWE: Full weakness classifications, likelihood of exploit, and abstractions.
- Total Dataset Size: ~187,700 structured instruction samples.
Evaluation Metrics
The fine-tuned model was evaluated against an unseen hold-out test dataset to mathematically verify its understanding of cybersecurity concepts and generation quality.
| Metric | Score | Interpretation |
|---|---|---|
| Perplexity | 7.61 | Excellent. Reflects high confidence and deep vocabulary retention for security concepts. |
| METEOR | 0.4084 | Very Good. The model captures semantic meaning effectively, correctly utilizing security synonyms. |
| ROUGE-1 | 0.3496 | High structural and unigram overlap with security researcher standards. |
| ROUGE-L | 0.2044 | Consistent sentence-level alignment for technical vulnerability reports. |
Usage & Inference
To load the model quickly using Unsloth for 2x faster inference:
from unsloth import FastLanguageModel
# Load the model directly from this Hugging Face repository
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="Mohamedabul/Qwen2.5-3B-CyberSec-Instruct", # or your exact repo name
max_seq_length=1024,
dtype=None,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
# Example Prompt
instruction = "Analyze this vulnerability: CVE-2021-44228 (Log4Shell). Provide attack vectors, severity, and mitigation."
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": instruction}], tokenize=False, add_generation_prompt=True
)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])