Character similarity between images
CCIP enables determining whether two images depict the same character, calculating visual similarity between characters.
There are up to 6000 (female) characters available.
The model assigns a score to each image patch to make the features distinguishable on a 2D plane.
The cosine similarity between the scores of two images is used to identify the same characters in both.
The takeaway is that the model doesn't need a tokenizer, text model, or cross-attention. The labels are unknown to the model during training.
Source data
- live gelbooru feed
- character_index
- character_similarity
References
- Yann LeCun's $1B Bet Against LLMs [Welch Labs]