CyberNeurova · Qwen2.5-VL-3B-Instruct · Abliterated
CyberNeurova research — cyberneurova.ai. First VLM release in the CyberNeurova abliteration line.
A permanently-abliterated version of
Qwen/Qwen2.5-VL-3B-Instruct.
Refusal behavior has been removed via residual-stream weight orthogonalization
(Arditi et al. 2024 family of techniques). Inference is unchanged — no runtime
hooks, no slowdown, no custom code. Both text-only and image+text
modes are unlocked from a single capture.
Status: experimental research artifact. This is for defensive security research, red-team evaluation, and academic study of refusal mechanisms in multimodal models. Use at your own discretion.
Headline results
Measured on the bf16 hooked model:
| Probe | Baseline | Abliterated | Δ |
|---|---|---|---|
| refusal_rate (AdvBench-style, 29 prompts) | 86.2% | 3.4% | −82.8 pp |
| soft_refusal_probe (55 hard OOD prompts) | 81.8% | 5.5% | −76.4 pp |
| multimodal_probe (5 image+harmful) | 100.0% | 0.0% | −100.0 pp |
| perplexity (wikitext-2) | 9.52 | 9.39 | −0.13 (coherence preserved) |
| distinct-2 diversity | 0.63 | 0.82 | +0.20 (output diversity ↑) |
The standout result: perplexity went DOWN and diversity went UP. No visible "abliteration tax" on the language metrics — the refusal direction was unusually clean.
Security & capability benchmarks
Full 8-benchmark suite run via vLLM on the bf16 hooked model (baseline + ablated back-to-back per benchmark). Capability scores measure both compliance (does it engage with the prompt vs refuse) and technical specificity (does it produce technically substantive text). Refusal is the only "lower-is-better" benchmark; everything else is higher-is-better.
| Benchmark | Direction | Baseline | Abliterated | Δ |
|---|---|---|---|---|
| refusal | lower better | 93.9% | 6.1% | −87.8 pp |
| hacking (pen-testing / exploits) | higher better | 24.3% | 56.0% | +31.7 pp |
| cyber_weapons (malware / ransomware / C2) | higher better | 39.0% | 49.3% | +10.3 pp |
| bug_finding (defensive code review) | higher better | 35.0% | 41.7% | +6.7 pp |
| reasoning (math/logic) | higher better | 33.3% | 36.7% | +3.4 pp |
| tool_calling (JSON function calls) | higher better | 93.1% | 93.1% | 0.0 |
| coding (HumanEval-style) | higher better | 6.7% | 6.7% | 0.0 |
| coherence (open-ended fluency) | higher better | 83.7% | 79.9% | −3.8 pp |
The standout: hacking went from 24% → 56% — a +31.7 pp jump. That benchmark scores both whether the model engages with pen-testing prompts and whether it produces technical specifics rather than hand-waving. The abliterated model meaningfully unlocks practical offensive-security content the original was over-cautious about. Defensive capability (bug_finding) also improves, indicating the original safety training was over-blocking neutral security topics too.
Compliance vs competence — important caveat. This is a 3B model. It will engage with almost any cyber prompt post-ablation, but its technical depth is shallow: coding stays at 6.7% (HumanEval is hard for 3B models), and cyber-weapons scoring tops out at 49% because the model doesn't always know the correct technical details to follow up with. For research-grade red-team baselines and refusal-mechanism studies this is exactly the right tool; for actual offensive capability evaluation, use a 7B+ or 32B+ model.
How it works (one paragraph)
Modern VLMs like Qwen2.5-VL have two halves: a vision encoder (turns pixels into tokens) and a language model (predicts the next token given text + vision tokens). Safety RLHF lives in the language model's residual stream — when the model refuses, it's an LM-side behavior, not a vision-side one. So removing a single refusal representation from the LM tower flips both text-only AND image-conditioned refusals at the same time, even when the capture itself only used text prompts. One direction, two modes unlocked.
How to download
hf download cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated --local-dir ./Qwen2.5-VL-3B-abl
~7 GB safetensors + tokenizer + processor.
How to run
Text-only (chat)
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
"cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated",
torch_dtype=torch.bfloat16, device_map="cuda:0",
)
processor = AutoProcessor.from_pretrained("cyberneurova/CyberNeurova-Qwen2.5-VL-3B-Instruct-abliterated")
messages = [{"role": "user", "content": [{"type": "text", "text": "Your prompt here."}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])
Multimodal (image + text)
from PIL import Image
img = Image.open("your_image.jpg")
messages = [{"role": "user", "content": [
{"type": "image", "image": img},
{"type": "text", "text": "Describe what's in this image."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[img], return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])
Hardware requirements
| bfloat16 (this release) | int4 (with bitsandbytes) | |
|---|---|---|
| File size | 7.5 GB | ~2.5 GB |
| VRAM (minimum) | 8 GB | 4 GB |
| Recommended | 12 GB+ for long contexts | 8 GB |
Runs comfortably on a single 4090 / 6000 / A6000 / H100 / consumer Blackwell.
Intended use
Defensive security research, jailbreak-evaluation baselines, multimodal
safety-mechanism research, and academic study of how refusal directions
behave in VLMs. Useful as a counterfactual against the original
Qwen/Qwen2.5-VL-3B-Instruct for measuring the precise behavioral impact
of safety RLHF.
Not intended for automating harmful action. The abliteration removes canonical refusal behavior but does not remove the model's underlying knowledge — the model still recognises harmful instructions as harmful, it simply no longer refuses them by pattern. The model's competence on harmful technical content is also limited by its 3B parameter count (Qwen2.5-VL-3B is a small VLM and frequently gets technical details wrong even when it tries to comply).
Limitations
- A small fraction of prompts still produce refusals (~3-5% on AdvBench-style, TBD on harder OOD probes). Linear residual-stream ablation cannot remove the long tail without quality damage.
- The model is a 3B VLM. Compliance ≠ competence. When asked harmful technical content (chemistry, malware, weapons), the model may produce factually incorrect or oversimplified answers because it just doesn't know the detailed answer. This is a model-scale limitation, not abliteration.
- The vision encoder is unchanged. If you're researching vision-side safety features (image-classifier filters), this release won't help — those live outside the LM residual stream.
- Long-context (>4k tokens) behavior post-abliteration is not validated in this release.
License
Apache 2.0 (inherits from upstream Qwen2.5-VL-3B-Instruct).
Acknowledgements
- Alibaba Qwen for Qwen2.5-VL-3B-Instruct
- Arditi et al. 2024 for the refusal-direction methodology this work builds on
- ByteDance Lance for the case study that prompted us to build VLM support into our abliteration framework