AD-Copilot
Comparison-aware anomaly detection with vision-language models. Extends Qwen2.5-VL-7B with a novel comparison-aware visual encoder achieving 78.74% on OmniDiff benchmark.
Key Innovation
- ADCopilotCompareVisualEncoder: Bidirectional cross-attention comparing reference and test images
- 100 comparison tokens per image pair injected into the language model
- State-of-the-art on industrial anomaly detection benchmarks
Quick Start
import torch
from transformers import AutoModelForVision2Seq, AutoProcessor
from qwen_vl_utils import process_vision_info
model = AutoModelForVision2Seq.from_pretrained(
"jiang-cc/AD-Copilot",
torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
"jiang-cc/AD-Copilot",
min_pixels=64*28*28, max_pixels=1280*28*28, trust_remote_code=True,
)
messages = [{"role": "user", "content": [
{"type": "image", "image": "good.png"},
{"type": "image", "image": "test.png"},
{"type": "text", "text": "The first image is good. Is there any anomaly in the second image? A.yes, B.no."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, _ = process_vision_info(messages)
inputs = processor(text=[text], images=[image_inputs], return_tensors="pt").to(model.device)
with torch.inference_mode():
ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
trimmed = [o[len(i):] for i, o in zip(inputs.input_ids, ids)]
print(processor.batch_decode(trimmed, skip_special_tokens=True)[0])
Results (OmniDiff Benchmark)
| Model | Visited IAD | Avg ACC |
|---|---|---|
| MiniCPM-V2.6 | 0 | 67.90% |
| EIAD | 128k | 69.40% |
| Qwen2.5-VL | 0 | 72.19% |
| AD-Copilot (Ours) | 206k | 78.74% |
Architecture
- Base: Qwen2.5-VL-7B-Instruct (28 layers, 3584 hidden)
- Vision: Qwen2.5-VL ViT (32 layers, 1280 hidden)
- Comparison Encoder: Bidirectional cross-attention + query decoder (100 tokens)
- Dtype: bfloat16
Citation
@article{adcopilot2025,
title={AD-Copilot: Comparison-Aware Anomaly Detection with Vision-Language Models},
author={Jiang, Xi and others},
journal={arXiv preprint arXiv:2603.13779},
year={2025}
}