Robust perceptual metric, based on CLIP model laion/CLIP-ViT-B-32-laion2B-s34B-b79K
Adversarially fine-tuned with FARE (Schlarmann et al. (2024)) on ImageNet with infinity-norm and radius 4/255.
Performance on the perceptual similarity task NIGHTS:
Clean L-inf, eps=4/255 L2, eps=3
91.1 71.8 70.6
Usage
model, _, image_processor = open_clip.create_model_and_transforms('hf-hub:chs20/FARE4-ViT-B-32-laion2B-s34B-b79K')
Citation
If you find this model useful, please consider citing our papers:
@inproceedings{croce2024adversarially,
title={Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics},
author={Croce, Francesco and Schlarmann, Christian and Singh, Naman Deep and Hein, Matthias},
year={2025},
booktitle={{SaTML}}
}
@inproceedings{schlarmann2024robustclip,
title={Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models},
author={Schlarmann, Christian and Singh, Naman Deep and Croce, Francesco and Hein, Matthias},
year={2024},
booktitle={{ICML}}
}