🛡️ Qwen2.5-7B Phishing Threat Engine (16-bit Merged)
This repository contains the fine-tuned, 16-bit merged weights (BF16) of Qwen2.5-7B-Instruct, optimized for enterprise email threat classification, social engineering detection, and zero-link BEC (Business Email Compromise) analysis.
Trained using Unsloth and QLoRA, this model accurately classifies incoming email payloads as either 'Phishing' or 'Benign' while preserving structural context and minimizing false positives on legitimate corporate communications.
This model is a fine-tuned version of Qwen/Qwen2.5-7B, originally released by Alibaba Cloud under the Apache 2.0 License
📊 Evaluation & Benchmark Results
Evaluated on a held-out, deduplicated unseen test split ($N = 500$ emails) consisting of real-world corporate communications, spear-phishing lures, and false-positive traps:
| Model Variant | Tuned Target Modules | Accuracy | Phishing F1-Score | ROC-AUC | Avg. Latency | VRAM Footprint |
|---|---|---|---|---|---|---|
| Qwen2.5-7B (Base Zero-Shot) | None | 82.40% | 80.10% | 0.8310 | ~240 ms | 14.2 GB (16-bit) |
| Standard LoRA | q, k, v, o | 97.00% | 96.50% | 0.9681 | ~250 ms | 5.8 GB (4-bit AWQ) |
| Comprehensive LoRA (Ours - Best) | q, k, v, o, gate, up, down | 97.80% | 97.46% | 0.9773 | ~255 ms | 5.9 GB (4-bit AWQ) |
Key Capabilities
- Zero-Link BEC Detection: Identifies payroll routing diversions, vendor bank details changes, and urgent executive gift-card lures lacking links or attachments.
- Low False-Positive Rate: Distinguishes legitimate MFA reset codes, AWS billing threshold alerts, Okta lockouts, and bug bounty reports from malicious lures.
- Structured Output Ready: Trained on ChatML format for seamless integration with vLLM guided decoding (xgrammar/Pydantic v2).
🚀 Quickstart: Running Inference
Option 1: Native Hugging Face transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Ilieg/qwen2.5-7b-phishing-standard-merged-16bit"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
system_prompt = "You are an expert cybersecurity email threat analyzer. Classify the email as either 'Phishing' or 'Benign'."
user_message = "Subject: URGENT: Direct Deposit Update Required\n\nPlease update my account routing number immediately prior to Friday payroll."
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=10, do_sample=False)
prediction = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True).strip()
print(f"Classification: {prediction}")