MODA Fashion Distilled 512D — Vision Only

A lightweight vision-only checkpoint derived from HopitAI/moda-fashion-distilled-512d, packaged for fashion image-to-image retrieval, product matching, and visual similarity search.

This repository removes checkpoint parameters that are not required for image-only inference while retaining the complete visual encoder and the learned MODA 768 → 512 projection.

The retained checkpoint remains in FP32 and is distributed in Safetensors format.

Model Overview

The model converts a fashion image into a 512-dimensional L2-normalized embedding suitable for cosine-similarity search and vector databases.

Input Image
    ↓
ViT-B/16 SigLIP Visual Encoder
    ↓
768-D Visual Feature
    ↓
MODA Linear Projection
768 → 512
    ↓
L2 Normalization
    ↓
512-D Fashion Embedding

Checkpoint contents

This vision-only checkpoint contains:

• 162 visual.* tensors
• 1 proj.weight tensor
• proj.weight shape: [512, 768]
• Approximately 93.3 million retained parameters
• FP32 precision
• Safetensors format
• Approximately 373 MB on Hugging Face

The checkpoint does not include the text-side weights required for text encoding.

Intended Use

This model is designed for:

• Fashion image similarity search
• Image-to-image retrieval
• Similar product search
• Product matching
• Fashion catalog indexing
• Duplicate and near-duplicate detection
• Visual recommendations
• Fashion product clustering
• Vector database embeddings
• Visual search APIs

Not supported

Because the text encoder weights are not included, this checkpoint does not support:

• Text-to-image search
• Text embeddings
• Natural-language search queries
• Zero-shot text classification

This repository is specifically intended for image-to-image applications.

Architecture

Installation

Install the required packages:

pip install torch open_clip_torch safetensors pillow huggingface_hub

Quick Start

import torch
import torch.nn as nn
import torch.nn.functional as F
import open_clip

from PIL import Image
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file


REPO_ID = "manoj45232/moda-fashion-distilled-512d-vision-only"


# Download checkpoint
model_path = hf_hub_download(
    repo_id=REPO_ID,
    filename="model.safetensors"
)


# Build the matching OpenCLIP architecture
model, _, preprocess = open_clip.create_model_and_transforms(
    "ViT-B-16-SigLIP",
    pretrained=None
)


# Load the vision-only MODA checkpoint
state = load_file(model_path)

# Extract the learned MODA 768 -> 512 projection
proj_weight = state.pop("proj.weight")

# Text-side weights are intentionally absent, so strict=False is required
missing, unexpected = model.load_state_dict(
    state,
    strict=False
)


# Restore the MODA projection
proj = nn.Linear(
    768,
    512,
    bias=False
)

with torch.no_grad():
    proj.weight.copy_(proj_weight)


model.eval()
proj.eval()


# Load an image
image = Image.open("fashion_item.jpg").convert("RGB")
image_tensor = preprocess(image).unsqueeze(0)


# Generate normalized 512-D embedding
with torch.no_grad():
    visual_features = model.encode_image(image_tensor)
    embedding = proj(visual_features)
    embedding = F.normalize(
        embedding,
        p=2,
        dim=-1
    )


print("Embedding shape:", embedding.shape)
print("Embedding norm:", embedding.norm(dim=-1))

# Expected:
# torch.Size([1, 512])

GPU Inference

Use CUDA automatically when available:

device = "cuda" if torch.cuda.is_available() else "cpu"

model = model.to(device)
proj = proj.to(device)
image_tensor = image_tensor.to(device)

with torch.no_grad():
    features = model.encode_image(image_tensor)
    embedding = F.normalize(
        proj(features),
        p=2,
        dim=-1
    )

Reusable Image Encoder

def encode_image(image_path):
    image = Image.open(image_path).convert("RGB")
    tensor = preprocess(image).unsqueeze(0).to(device)

    with torch.no_grad():
        features = model.encode_image(tensor)
        embedding = proj(features)
        embedding = F.normalize(
            embedding,
            p=2,
            dim=-1
        )

    return embedding

Example:

embedding = encode_image("shoe.jpg")
print(embedding.shape)

Expected output:

torch.Size([1, 512])

Image Similarity

Because the embeddings are L2-normalized, cosine similarity can be calculated using a dot product:

embedding_a = encode_image("shoe_1.jpg")
embedding_b = encode_image("shoe_2.jpg")

similarity = embedding_a @ embedding_b.T

print("Similarity:", similarity.item())

A larger score generally indicates greater similarity in the learned fashion embedding space.

Batch Inference

For larger catalogs, process multiple images together:

from PIL import Image
import torch
import torch.nn.functional as F


paths = [
    "item1.jpg",
    "item2.jpg",
    "item3.jpg",
]

batch = torch.stack([
    preprocess(Image.open(path).convert("RGB"))
    for path in paths
]).to(device)

with torch.no_grad():
    features = model.encode_image(batch)
    embeddings = F.normalize(
        proj(features),
        p=2,
        dim=-1
    )

print(embeddings.shape)

# torch.Size([3, 512])

Vector Database Usage

Each image produces one 512-dimensional normalized vector.

These embeddings can be stored in vector-search systems such as:

• FAISS
• Qdrant
• Milvus
• Weaviate
• Pinecone
• Elasticsearch
• OpenSearch
• PostgreSQL with pgvector

A typical production search pipeline is:

Catalog Images
      ↓
MODA Vision-Only
      ↓
512-D Embeddings
      ↓
Vector Database
      ↑
512-D Query Embedding
      ↑
Customer Query Image

At search time:

1. Encode the query image.
2. Generate its normalized 512-D embedding.
3. Search the vector database.
4. Rank catalog vectors by cosine similarity.
5. Return the highest-ranking fashion products.

Why Vision Only?

For pure image-to-image retrieval, the text encoder is not required.

This repository retains:

visual.*
proj.weight

The resulting image-only pipeline is:

Image
  ↓
ViT-B/16 SigLIP
  ↓
768-D
  ↓
MODA Projection
  ↓
512-D
  ↓
L2 Normalization

Important Loading Note

This checkpoint intentionally does not contain a complete OpenCLIP multimodal state dictionary.

Do not use:

model.load_state_dict(state, strict=True)

because the text-side weights are intentionally absent.

Use:

model.load_state_dict(state, strict=False)

Missing text-side keys are expected for this vision-only checkpoint.

Preprocessing

For best compatibility, use the preprocessing returned by:

open_clip.create_model_and_transforms(
    "ViT-B-16-SigLIP",
    pretrained=None
)

Avoid changing preprocessing unless you intentionally want different model behavior.

Important preprocessing properties include:

• Image resolution
• Resize behavior
• Crop behavior
• Pixel normalization
• RGB conversion

Different preprocessing can produce different embeddings and may affect retrieval quality.

Embedding Normalization

Always apply L2 normalization after the MODA projection:

embedding = F.normalize(
    proj(features),
    p=2,
    dim=-1
)

This produces unit-length embeddings suitable for cosine similarity.

Download Only the Model File

Python:

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="manoj45232/moda-fashion-distilled-512d-vision-only",
    filename="model.safetensors"
)

print(model_path)

Hugging Face CLI:

hf download manoj45232/moda-fashion-distilled-512d-vision-only model.safetensors

Production Deployment

For production deployments, download and initialize the model once when the application starts.

A typical deployment architecture is:

Application Startup
       ↓
Download model.safetensors
       ↓
Load SigLIP visual model
       ↓
Load MODA projection
       ↓
Move model to CPU/GPU
       ↓
Keep model in memory
       ↓
Serve image embedding requests

The model can be integrated into:

• FastAPI
• Flask
• Django
• Docker
• Kubernetes
• GPU inference services
• Product indexing workers
• Search APIs
• Recommendation systems

Model Relationship

This repository is a derived packaging of:

HopitAI/moda-fashion-distilled-512d

The trained visual representations and learned projection originate from the upstream MODA project.

This repository does not claim to have trained the underlying model.

The modification is limited to packaging the checkpoint for vision-only image retrieval.

Changes From the Upstream Checkpoint

Retained

visual.*
proj.weight

This checkpoint contains:

162 visual tensors
+
1 projection tensor
=
163 tensors total

The projection tensor has shape:

[512, 768]

Removed

Parameters not required by the image-only embedding path are not included.

No FP16 conversion or quantization was applied.

Performance

This repository is intended to preserve the visual embedding pathway of the upstream MODA checkpoint.

No independent retrieval benchmark is currently reported for this derivative repository.

For benchmark results and evaluation methodology, refer to the original model:

HopitAI/moda-fashion-distilled-512d

Users should evaluate retrieval performance on their own catalog and query distribution before production use.

Limitations

Performance may be lower for:

• Non-fashion imagery
• Very low-resolution images
• Heavily occluded products
• Unusual or out-of-distribution fashion categories
• Search tasks driven mainly by textual semantics
• Images significantly different from the upstream training distribution

Retrieval quality can also be affected by:

• Incorrect preprocessing
• Poor image quality
• Background clutter
• Vector database configuration
• Similarity metric
• Catalog image consistency
• Product photography style

Checkpoint File

The main checkpoint is:

model.safetensors

It contains the visual-side model parameters and learned MODA 512-D projection.

Safetensors

The checkpoint is distributed using the Safetensors format.

Safetensors stores tensor data without relying on Python pickle execution.

License

This repository uses the MIT License, following the upstream model repository.

Users should also review the upstream model and project documentation before production or commercial use.

Attribution

Original model:

HopitAI/moda-fashion-distilled-512d

Hugging Face:

https://huggingface.co/HopitAI/moda-fashion-distilled-512d

MODA project:

https://github.com/hopit-ai/Moda

If you use this checkpoint, please credit the original MODA / HopitAI authors.

Acknowledgements

Thanks to the MODA and HopitAI authors for releasing the original fashion retrieval model.

This repository provides a smaller vision-only packaging of the upstream trained visual model for image-to-image retrieval and deployment.