See our collection for all versions of SigLIP.

Run SigLIP with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/siglip_base_p16_multilingual_256

Paper: Sigmoid Loss for Language Image Pre-Training (arXiv:2303.15343) · HF Papers

SigLIP is a vision + text dual encoder trained with a pairwise sigmoid loss instead of CLIP's softmax contrastive loss. Per-pair training scales to large batches and often improves zero-shot accuracy at the same model size.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of google/siglip-base-patch16-256-multilingual for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a zero-shot image-text checkpoint (SigLIPZeroShotClassify): pass image(s) and text prompts at inference time.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.siglip import (
    SigLIPProcessor,
    SigLIPZeroShotClassify,
)

processor = SigLIPProcessor.from_weights("zeromodels/siglip_base_p16_multilingual_256")
model = SigLIPZeroShotClassify.from_weights("zeromodels/siglip_base_p16_multilingual_256")

labels = [
    "a photo of a cat",
    "a photo of a dog",
    "a photo of a car",
    "a photo of a living room",
]
inputs = processor(text=labels, image_paths="your_image.jpg")
output = model(
    {
        "images": inputs["images"],
        "token_ids": inputs["input_ids"],
    }
)
print(output["image_logits"].shape)

Load any SigLIP variant the same way with from_weights("zeromodels/<variant>"):

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Prefer Processor.from_weights(...) so image size and tokenizer match the variant.
• Map processor input_ids to model token_ids. No padding mask is required.
• See SigLIP docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. SigLIPZeroShotClassify.from_weights("hf:google/siglip-base-patch16-256-multilingual").

Special Thanks

A huge thank you to the Google SigLIP authors for creating and releasing these models.

License: Apache 2.0.