MobileSAM: Optimized for AMD ROCm
MobileSAM is a lightweight, promptable segmentation model — a distilled Segment Anything Model (SAM) that predicts a pixel-level mask for any object from a single point prompt. This repository packages inference for promptable instance segmentation using ONNX Runtime, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs, CPUs, and NPUs.
This is based on the implementation of MobileSAM found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the mobilesam AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.
Task Overview
Task: Promptable instance segmentation (point prompt → mask)
Dataset: SA-1B subset (10 images, per-mask point-prompt annotations)
Output metrics: mIoU, Mean Mask IoU, Median Mask IoU, IoU Std Dev, Min IoU, Max IoU
AMD ROCm Optimization
This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points:
- Validated backends: ONNX Runtime across CPU (FP32), GPU (MIGraphX execution provider — FP32/FP16 for benchmarking, FP32/FP16/BF16/INT8 for profiling and evaluation), and NPU (VitisAI execution provider, auto-quantized internally).
- The pipeline is a two-stage ONNX graph — a TinyViT image encoder (once per image) feeding a prompt/mask decoder (once per point).
- No code changes required versus the upstream MobileSAM implementation — only environment/runtime configuration differs.
- On GPU, the MIGraphX execution provider writes the TinyViT encoder output in NHWC memory order; the evaluator reinterprets it back to NCHW so GPU accuracy matches CPU.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| ONNX Runtime | FP32 | CPU Execution Provider | AMD CPU | — |
| ONNX Runtime | FP32 / FP16 (benchmark); FP32/FP16/BF16/INT8 (eval/profile) | MIGraphX Execution Provider | AMD Instinct™ / Radeon™ GPU (ROCm) | NHWC output reinterpreted to NCHW |
| ONNX Runtime | Auto | VitisAI Execution Provider | AMD Ryzen AI NPU | Auto-quantized internally; quality lower than CPU/GPU |
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the mobilesam on GitHub.
Model Details
Model Type: Promptable instance segmentation — distilled Segment Anything Model with a TinyViT encoder
Base Model: MobileSAM (pinned commit b01a9ccef3b9e10b099b544efe004d0871802c3b)
Model Stats:
- Two-stage pipeline: TinyViT image encoder + prompt/mask decoder
- Encoder input:
(1, 3, 1024, 1024)float32; encoder output (image_embeddings):(1, 256, 64, 64)float32 - Decoder input (point_coords):
(1, 1, 2)float32; (point_labels):(1, 1)float32 - Decoder output (low_res_masks):
(1, 1, 256, 256)float32; (iou_predictions):(1, 1)float32 - Precision tested: FP32 (CPU); FP32, FP16, BF16, INT8 (GPU); auto-quantized (NPU)
Accuracy Pipeline
Higher mIoU means the model's predicted masks overlap more closely with the ground-truth annotations — 1.0 would be pixel-perfect agreement, 0.0 means no overlap. In practice, values above ~0.5 for mIoU on SA-1B are considered strong for prompted segmentation.
Metrics Explained
| Metric | Description |
|---|---|
| mIoU | Primary metric — mean Intersection over Union averaged across all images. Higher means predicted masks align closely with ground truth across the evaluation set. |
| Mean Mask IoU | Average per-mask IoU across all individual point prompts — captures how well the model handles each mask independently, not just the per-image average. |
| Median Mask IoU | Median per-mask IoU — robust to outliers; if this is much higher than Mean Mask IoU, a few bad masks are dragging the mean down. |
| IoU Std Dev | Standard deviation of per-mask IoU scores — low std means consistent quality across all prompts; high std means some masks are much worse than others. |
| Min IoU / Max IoU | Range of per-mask IoU — exposes the worst-case and best-case mask predictions. A very low Min IoU indicates the model occasionally produces completely wrong masks. |
Accuracy Results
Full Dataset Evaluation (SA-1B subset, 10 images) — filled from runs/quality_<device>_<precision>.json; run make metrics to refresh:
| Device | Precision | mIoU | Mean Mask IoU | Median Mask IoU | IoU Std |
|---|---|---|---|---|---|
| CPU | FP32 | 0.5752 | 0.5752 | 0.5711 | 0.0574 |
| GPU | FP32 | 0.5752 | 0.5752 | 0.5711 | 0.0574 |
| GPU | FP16 | 0.5750 | 0.5750 | 0.5681 | 0.0563 |
| NPU | Auto | 0.4252 | 0.4252 | 0.4174 | 0.0753 |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for CPU, GPU, and NPU runners (encoder + decoder benchmarking/profiling)
- SA-1B dataset staging and mIoU evaluation pipeline
- Benchmarking and reproduction instructions