InSPyReNet ONNX (Swin-B 1024x1024)
An ONNX export of InSPyReNet (Incongruent Salient Object Detection) using the Swin-B transformer backbone at 1024x1024 resolution. This model outputs fine-grained alpha mattes for zero-shot background removal, hair matting, and salient object extraction.
The model file is self-contained (all initializers and weights are embedded directly in the 357 MB file). You can run it on CPU, CUDA, Apple Silicon via CoreML, or inside web browsers with WebGPU through ONNX Runtime.
Model Summary
| Property | Value |
|---|---|
| Architecture | InSPyReNet (Swin-B backbone) |
| Task | Salient Object Detection / Background Removal |
| Format | ONNX (IR version 10, Opset 17) |
| Available Variants | FP32 (357 MB), FP16 (183 MB), INT8 (99 MB), UINT8 (98 MB), Q4 (71 MB) |
| Input Tensor | input: [1, 3, 1024, 1024] (Float32, RGB) |
| Output Tensor | alpha: [1, 1, 1024, 1024] (Float32, range [0.0, 1.0]) |
| Preprocessing | Resize to 1024x1024, scale to [0, 1], ImageNet mean/std |
| License | Apache 2.0 (Upstream InSPyReNet is MIT) |
Available Quantizations
| File | Precision | File Size | Recommended Runtime | Notes |
|---|---|---|---|---|
model.onnx | Float32 | 357 MB | High-precision reference | Original full-precision export |
model_fp16.onnx | Float16 | 183 MB | CUDA, WebGPU, Apple Silicon | Half size, faster execution, virtually lossless alpha edges |
model_int8.onnx | Dynamic INT8 | 99 MB | Multi-core CPU servers | ~72% smaller, fast linear layer execution on CPU |
model_uint8.onnx | Dynamic UINT8 | 98 MB | Specific CPU runtimes | Unsigned 8-bit dynamic quantization |
model_q4.onnx | 4-bit block-wise | 71 MB | Browser, mobile, edge memory | ~80% smaller, optimal for client-side download |
Quickstart (Python)
Install onnxruntime and pillow:
pip install onnxruntime pillow numpy
# Or for GPU support:
# pip install onnxruntime-gpu pillow numpy
Run background removal in Python:
import numpy as np
import onnxruntime as ort
from PIL import Image
# 1. Load image and keep original dimensions
img = Image.open("input.jpg").convert("RGB")
orig_w, orig_h = img.size
# 2. Resize to 1024x1024 and normalize
resized = img.resize((1024, 1024), Image.Resampling.BILINEAR)
arr = np.array(resized, dtype=np.float32) / 255.0
mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
norm = (arr - mean) / std
# 3. Shape to [1, 3, 1024, 1024]
tensor = np.transpose(norm, (2, 0, 1))[np.newaxis, ...]
# 4. Run inference (use model_fp16.onnx, model_int8.onnx, or model_q4.onnx as needed)
session = ort.InferenceSession("model_fp16.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
out = session.run(None, {"input": tensor})[0]
# 5. Extract alpha and resize back to original size
alpha = (np.clip(out[0, 0], 0.0, 1.0) * 255.0).astype(np.uint8)
alpha_img = Image.fromarray(alpha, mode="L").resize((orig_w, orig_h), Image.Resampling.BILINEAR)
# 6. Save transparent cutout
cutout = img.convert("RGBA")
cutout.putalpha(alpha_img)
cutout.save("cutout.png")
CLI Usage
This repository includes a ready-to-use Python script in examples/infer.py:
# Process a single image to a transparent PNG
python examples/infer.py --image photo.jpg --output cutout.png
# Process a single image on CUDA
python examples/infer.py --image photo.jpg --provider cuda
# Save only the grayscale alpha matte mask
python examples/infer.py --image photo.jpg --mask-only --output mask.png
# Batch process a folder of images
python examples/infer.py --dir path/to/images --output-dir path/to/cutouts
Browser Usage (WebGPU & WASM)
You can run this model directly on client devices inside the browser using onnxruntime-web.
A complete demo application is provided in examples/index.html. To test it locally:
cd examples
python3 -m http.server 8000
Open http://localhost:8000 in your browser. You can load the model via WebGPU or WASM, drag and drop any image, view the live alpha mask, and download transparent PNG cutouts.
Minimal JavaScript snippet:
import * as ort from "onnxruntime-web";
// Initialize session with WebGPU
const session = await ort.InferenceSession.create("inspyrenet_swinb_1024.onnx", {
executionProviders: ["webgpu"]
});
// Prepare Float32Array tensor with shape [1, 3, 1024, 1024]
// normalized with ImageNet mean/std
const feeds = { input: inputTensor };
const results = await session.run(feeds);
const alphaData = results.alpha.data; // Float32Array of 1024 * 1024 alpha values
Technical Notes
- Input resolution: The model expects an input tensor of
1x3x1024x1024. Non-square images should be resized to 1024x1024 for inference, and the resulting alpha mask resized back to the original image dimensions. - Output range: The output values represent foreground probability between
0.0(background) and1.0(foreground). Soft edges (such as hair or transparent objects) contain fractional values between0.0and1.0. - Hardware acceleration: On modern desktop GPUs (RTX series, Apple Silicon M-series), inference runs in 50 to 120 ms. On multi-core CPUs, inference typically completes in 500 to 1200 ms depending on thread count.
Upstream Research & Attribution
This model is derived from the research project:
- Paper: InSPyReNet: Incongruent Salient Object Detection (ACM MM 2023)
- Authors: Taehun Kim, Kunhee Kim, Joonyeong Lee, Dongmin Cha, Jiho Lee, Daijin Kim
- Upstream Code: github.com/plemeri/InSPyReNet
License
The files in this repository are licensed under the Apache 2.0 License. See LICENSE for details. Upstream research code for InSPyReNet is licensed under the MIT License.