DreamSim (ensemble) — HuggingFace format
Unofficial HuggingFace-format port of the DreamSim perceptual-similarity metric
(ensemble variant). Loads via trust_remote_code with no dreamsim / peft runtime
dependency; weights are the LoRA-merged ensemble stored as model.safetensors.
Not an official DreamSim release. DreamSim was created by Shobhita Sundaram, Netanel Tamir, Stephanie Fu, and Richard Zhang — see the original repository https://github.com/ssundaram21/dreamsim and paper "DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data" (NeurIPS 2023). This repo re-packages their released ensemble for
transformers.
What it is
The ensemble concatenates the features of three ViT-B/16 backbones — DINO (cls),
CLIP (embedding), OpenCLIP (embedding) — each LoRA-adapted, then mean/L2-normalizes
(embed_size = 1792). Perceptual distance between two images is 1 − cos.
This port has been verified against the official dreamsim package on 1,000 real
images: max |Δ embedding| < 3e-7 and max |Δ distance| < 2e-7 — identical up to
float32 rounding.
Usage
import torch
from PIL import Image
from transformers import AutoModel, AutoImageProcessor
model = AutoModel.from_pretrained("bigshanedogg/dreamsim", trust_remote_code=True).eval()
proc = AutoImageProcessor.from_pretrained("bigshanedogg/dreamsim", trust_remote_code=True)
img_a = proc(images=Image.open("a.png"), return_tensors="pt")["pixel_values"]
img_b = proc(images=Image.open("b.png"), return_tensors="pt")["pixel_values"]
emb = model(pixel_values=img_a).embeddings # (1, 1792)
dist = model.compute_distance(img_a, img_b) # 1 - cos (higher = more different)
The image processor resizes to 224×224 (BICUBIC) and scales to [0, 1]; the
per-backbone mean/std normalization is applied inside the model.
Licensing
- Port + original DreamSim code/weights: MIT — see
LICENSE. - Vendored ViT backbone code (adapted from DINO, Meta AI): Apache-2.0 — full text in
LICENSE.apache-2.0.txt; attribution + modification statement inNOTICE. - Merged backbone weights derive from DINO (Apache-2.0), CLIP (MIT), and OpenCLIP (MIT code, LAION-based weights). Review the upstream backbone terms — and, for OpenCLIP, the LAION data provenance — before any commercial use.