Model Card for HEURDU

This model is a duplicate heuristic mapper that attempts to speed up detection of similar images and determine if they are the same easily.

Model Details

Model Description

There are many ways to find the exact same image, but when you have 2 that are similar, then they become increasingly annoying to find differences. So, the goal here is to find those minor differences and point them out instead of count the bytes that are different. speeding up larger library duplicate detection.

The normal goal of duplicate detection is 2 exact images. This model solves the related goal of 2 images where 1 is a heavily compressed jpeg that someone cropped 1 pixel off instead.

• Developed by: [Yggdrasil75]
• Funded by [an empty wallet]

Uses

currently only usable via https://github.com/yggdrasil75/customimagemanager

The primary goal is to discern the difference between 2 photos of a person taking half a second apart when they smiled slightly differently. This is where a naive approach will find them too different (every pixel is off by a tiny amount) or an algorithmic approach (Phashing) will find them too similar.

Bias, Risks, and Limitations

This is not a 100% model, some verification still needed. It was trained mostly on real life, next version will be a larger dataset, including anime and so on. It will help you find similar images

How to Get Started with the Model

Check out my project at: https://github.com/yggdrasil75/customimagemanager to see how to use it.

Training Details

Training Data

Random public use photos from the internet.

Speeds, Sizes, Times

Medium is probably the best version for most users. xxl is the only one that beats it noticeably. I will be tuning the sizes to fit the data better.

Environmental Impact

My apartment was a little warmer this afternoon than this morning.

Proving it works:

Nebula werent in training data, You can easily automate removal of thumbnails from older media managers using this model. Where a naive approach or phashing will often miss overly compressed thumbnails, this should find them more accurately.