Model Card for Kiel-3-Cyber
Kiel-3-Cyber is a specialized domain-specific language model fine-tuned by KielTech for advanced application security, automated vulnerability analysis, and secure code auditing.
Model Details
Model Description
- Developed by: KielTech
- Model Type: Fine-tuned causal language model using Parameter-Efficient Fine-Tuning (QLoRA)
- Language(s): English
- Finetuned from:
unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bi - Primary Domain: Application security, vulnerability remediation, and secure code reviews (e.g., Python JWT validation, authentication logic, and threat surface analysis).
Uses
Direct Use
- Automated Code Audits: Scanning source code repositories for insecure coding patterns, injection flaws, and authorization bypasses.
- Security Refactoring: Providing production-ready, secure code patches to resolve identified vulnerabilities.
- Threat Reasoning: Evaluating system architecture and session management logic from a defensive engineering standpoint.
Out-of-Scope Use
- Kiel-3-Cyber is designed exclusively for defensive engineering, authorized security research, and educational code auditing. It should not be used to generate malicious payloads, exploit active infrastructure without authorization, or facilitate cyberattacks.
Get Started with the Model
You can load and query Kiel-3-Cyber using the Hugging Face transformers library:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "kiel2/Kiel-3-Cyber"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = """[Security Audit Request]
Analyze the following Python JWT session validation code for vulnerabilities:
```python
import jwt
def verify_session(token):
decoded = jwt.decode(token, "secret_key", algorithms=["HS256"])
return decoded
"""
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
Training Data
- Scale: Trained on a curated dataset comprising 15,000 domain-specific cybersecurity and secure coding samples.
- Focus Areas: Python security anti-patterns, JWT session validation, input sanitization, and cryptographic implementation standards.
Training Procedure
- Framework: Unsloth & TRL (
SFTTrainer) - Quantization: 4-bit precision base loading with QLoRA adaptation layers (r=16, lora_alpha=16).
- Hyperparameters:
- Learning Rate: 2 × 10⁻⁴
- Epochs: 1
- Optimizer:
adamw_8bit - Gradient Accumulation Steps: 8
Environmental Impact
- Hardware Type: NVIDIA GPU (Cloud-accelerated via Google Colab / Kaggle infrastructure)
- Compute Optimization: Utilized unsloth gradient checkpointing and memory-efficient training loops to minimize carbon footprint and VRAM utilization.
Citation
If you use Kiel-3-Cyber in your research, security workflows, or tooling, please credit KielTech:
@misc{kiel3cyber2026,
author = {KielTech},
title = {Kiel-3-Cyber: Specialized Cybersecurity Language Model for Automated Code Auditing},
year = {2026},
publisher = {Hugging Face},
journal = {Hugging Face Repository},
howpublished = {\url{[https://huggingface.co/kiel2/Kiel-3-Cyber](https://huggingface.co/kiel2/Kiel-3-Cyber)}}
}