Qwen3-4B-Cybersecurity
Qwen3-4B fine-tuned on 1.28M cybersecurity samples
🔪 Abliterated (refusal-removed) version: DexopT/Qwen3-4B-Cybersecurity-Heretic-16bit
Model Description
Qwen3-4B-Cybersecurity is a fine-tuned version of Qwen3-4B-Instruct-2507 specialized in cybersecurity topics including offensive security, penetration testing, vulnerability analysis, malware analysis, and threat intelligence.
Trained with Unsloth SFT on a curated dataset of 1,284,369 samples covering:
- 🔴 Offensive Security — SQL injection, XSS, buffer overflows, reverse shells
- 🔵 Defensive Security — CVE analysis, incident response, hardening
- 🟡 Network Security — MITM, ARP spoofing, DNS poisoning, DDoS
- 🟢 Active Directory — Kerberoasting, Pass-the-Hash, Golden Ticket attacks
- ⚫ Malware Analysis — Ransomware, rootkits, persistence techniques
- 🟣 Web Security — SSRF, XXE, CSRF, LFI/RFI, deserialization
Model Family
| Model | Description | Link |
|---|---|---|
| Qwen3-4B-Cybersecurity | Base fine-tuned model (this repo) | 📍 You are here |
| Qwen3-4B-Cybersecurity-Heretic-16bit | Abliterated — refusal directions removed | → |
| Qwen3-4B-Cybersecurity-GGUF | Q8_0 + Q4_K_M quantized for llama.cpp | → |
Training Details
| Parameter | Value |
|---|---|
| Base model | Qwen3-4B-Instruct-2507 |
| Training framework | Unsloth + TRL SFTTrainer |
| Training samples | 1,284,369 |
| Training steps | 800 |
| LoRA rank | 16 |
| Context length | 2048 |
| Batch size | 2 × 4 = 8 (effective) |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Optimizer | AdamW 8bit |
| Format | 16bit merged safetensors |
| Platform | Google Colab T4 |
Datasets
| Dataset | Samples | Description |
|---|---|---|
| DexopT/cyber_heretic | 1,284,369 | Raw unified cybersecurity dataset (JSONL) |
| DexopT/cyber_heretic_tokenized | 1,284,369 | Pre-tokenized Arrow dataset (1024 ctx) |
Dataset sources include .jsonl, .json, .csv, .parquet, and .arrow formats from multiple cybersecurity repositories, converted to unified OpenAI messages format.
Usage
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"DexopT/Qwen3-4B-Cybersecurity",
torch_dtype=torch.float16,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("DexopT/Qwen3-4B-Cybersecurity")
messages = [
{"role": "system", "content": "You are an expert cybersecurity assistant."},
{"role": "user", "content": "Explain how SQL injection works and how to prevent it."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.8)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
MLX (Apple Silicon)
pip install mlx-lm
mlx_lm.convert \
--hf-path DexopT/Qwen3-4B-Cybersecurity \
--mlx-path ~/models/qwen3-cyber-mlx \
--quantize --q-bits 8
mlx_lm.chat --model ~/models/qwen3-cyber-mlx
LM Studio / llama.cpp / Ollama
Use the GGUF version: DexopT/Qwen3-4B-Cybersecurity-GGUF
Refusal Behavior
This model retains some of the original refusal behavior from the base Qwen3 model. For a version with refusal directions removed via Heretic abliteration, see:
⚠️ Disclaimer
This model is intended for educational and research purposes only. Use responsibly and only on systems you have explicit permission to test. The authors are not responsible for any misuse.
Links
| 🔪 Heretic Version | DexopT/Qwen3-4B-Cybersecurity-Heretic-16bit |
| 📦 GGUF (Q8 + Q4) | DexopT/Qwen3-4B-Cybersecurity-GGUF |
| 📊 Training Dataset | DexopT/cyber_heretic |
| 🔧 Base Model | unsloth/Qwen3-4B-Instruct-2507 |
| 🏠 Qwen3 Collection | Qwen3 on HuggingFace |