See our collection for all versions of SAM.

Run SAM3 with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/sam3

Paper: SAM 3: Segment Anything with Concepts (arXiv:2511.16719) · HF Papers

SAM3 segments by concept, not location: give it a noun phrase and it finds every matching instance. A ViT-L backbone and FPN feed a DETR-style encoder/decoder with object queries; a CLIP text encoder supplies the open-vocabulary side. Boxes can still be mixed with text.

For more details on the model, please go to Meta's original model card.

Pure-Keras 3 conversion of facebook/sam3 for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a concept-prompted checkpoint (SAM3InstanceSegment / SAM3Detect / SAM3SemanticSegment): pass a text noun phrase (backbone ViT-L/14).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.sam3 import SAM3InstanceSegment

segmenter = SAM3InstanceSegment.from_weights("zeromodels/sam3")
result = segmenter.predict(
    images="your_image.jpg", text="person", threshold=0.3
)[0]
print(len(result["scores"]), result["masks"].shape)

Load any SAM / SAM2 / SAM3 variant the same way with from_weights("zeromodels/<variant>") (use SAM3InstanceSegment for this repo):

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• SAM / SAM2: point coordinates are in original pixel space; box prompts need enable_boxes=True / include_box_input=True when building the graph.
• SAM2 in this port is image-only (no video memory bank).
• SAM3: prefer SAM3InstanceSegment.predict(...) for text prompts; upstream facebook/sam3 is gated.
• See SAM3 docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. SAM3Model.from_weights("hf:facebook/sam3").

Special Thanks

A huge thank you to the Meta SAM 3 authors for creating and releasing these models.

License: see the SAM 3 LICENSE (Hub tag: other / sam-license). Upstream facebook/sam3 is gated.