SatMAE++ Transformers Models
Hugging Face–compatible checkpoints converted from the official SatMAE++ pretrained weights. Each subfolder is a standalone model repo layout (config.json, model.safetensors, preprocessor, and remote code) for feature extraction on FMoW satellite imagery.
Model Description
These models are ViT-Large encoders pretrained with multi-scale masked autoencoding (SatMAE++) on fMoW-RGB and fMoW-Sentinel imagery.
This collection currently bundles 2 converted pretrain checkpoints:
- FMoW-RGB: vanilla ViT-L/16, 3-channel BGR, 224×224
- FMoW-Sentinel: grouped-channel ViT-L/8, 10-band multispectral, 96×96
All folders ship self-contained remote code (modeling_satmae_pp.py, processor, pipeline) and load with trust_remote_code=True.
Developed by: techmn / SatMAE++
Converted for Hugging Face by: BiliSakura
License (weights): MIT
Original paper: Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery (CVPR 2024)
Available checkpoints
| Folder | Dataset | Encoder | Channels | Image | Patch | Legacy file |
|---|---|---|---|---|---|---|
satmae-pp-vit-large-patch16-fmow-rgb-pretrain | FMoW-RGB | vanilla ViT | 3 (BGR) | 224 | 16 | checkpoint_ViT-L_pretrain_fmow_rgb.pth |
satmae-pp-vit-large-patch8-fmow-sentinel-pretrain | FMoW-Sentinel | group-channel ViT | 10 | 96 | 8 | checkpoint_ViT-L_pretrain_fmow_sentinel.pth |
Legacy .pth filename mapping is in conversion_manifest.json.
Usage
Processors default to do_resize: false. Inputs keep native height and width; FMoW mean/std normalization still applies when enabled.
from transformers import pipeline
import numpy as np
REPO = "/path/to/SATMAE-PP-transformers"
SUBFOLDER = "satmae-pp-vit-large-patch8-fmow-sentinel-pretrain"
pipe = pipeline(
task="satmae-pp-feature-extraction",
model=REPO,
trust_remote_code=True,
model_kwargs={"subfolder": SUBFOLDER},
)
# FMoW-Sentinel: 10 bands at native size (e.g. 96×96 or larger)
image = np.random.randint(0, 255, (128, 128, 10), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape) # [1, 1024]
FMoW-RGB (BGR channel order):
SUBFOLDER = "satmae-pp-vit-large-patch16-fmow-rgb-pretrain"
pipe = pipeline(
task="satmae-pp-feature-extraction",
model=REPO,
trust_remote_code=True,
model_kwargs={"subfolder": SUBFOLDER},
)
image = np.random.randint(0, 255, (384, 384, 3), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape) # [1, 1024]
To resize to the pretraining reference (96×96 sentinel / 224×224 RGB):
features = pipe(image, pool=True, return_tensors=True, image_processor_kwargs={"do_resize": True})
Load components directly:
from transformers import AutoModel, AutoImageProcessor
model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
Normalization
The bundled image processor applies per-channel FMoW mean/std normalization by default (do_normalize=True). FMoW-RGB models expect BGR channel order; the processor swaps RGB→BGR when channel_order="bgr".
For FMoW-Sentinel, inputs should be 10-band reflectance arrays (bands 0, 9, 10 already dropped) with FMoW-Sentinel statistics baked into the preprocessor config.
Dependencies
transformers,torch,timm,safetensorsopencv-python(multispectral resize with more than 4 channels)
Citation
@inproceedings{satmaepp2024rethinking,
title={Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery},
author={Mubashir Noman and Muzammal Naseer and Hisham Cholakkal and Rao Muhammad Anwar and Salman Khan and Fahad Shahbaz Khan},
year={2024},
booktitle={CVPR}
}