SoftCon Transformers Models
Hugging Face–compatible checkpoints converted from the official SoftCon pretrained weights. Each subfolder is a standalone model repo layout (config.json, model.safetensors, preprocessor, and remote code) for feature extraction on Earth observation imagery.
Model Description
These models are encoders pretrained with multi-label guided soft contrastive learning on SSL4EO-S12 multispectral and SAR imagery, with DINOv2-style continual pretraining for ViT backbones.
This collection bundles 6 converted checkpoints:
- Architectures: ResNet-50, ViT-S/14, ViT-B/14 (DINOv2-style)
- Input modalities: S2-L1C 13-band (
s2c), S1 SAR 2-band (s1)
All folders ship self-contained remote code (modeling_softcon.py, processor, pipeline) and load with trust_remote_code=True. ViT backbones reuse transformers' built-in Dinov2Model.
Developed by: zhu-xlab / SoftCon
Converted for Hugging Face by: BiliSakura
License (weights): Apache-2.0
Original paper: Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining
Available checkpoints (6 models)
| Folder | Backbone | Modality | Channels | Embedding dim | Legacy file |
|---|---|---|---|---|---|
softcon-resnet50-s2c | RN50 | S2-L1C MS | 13 | 2048 | B13_rn50_softcon.pth |
softcon-vit-small-patch14-s2c | ViT-S/14 | S2-L1C MS | 13 | 384 | B13_vits14_softcon_enc.pth |
softcon-vit-base-patch14-s2c | ViT-B/14 | S2-L1C MS | 13 | 768 | B13_vitb14_softcon_enc.pth |
softcon-resnet50-s1 | RN50 | S1 SAR | 2 | 2048 | B2_rn50_softcon.pth |
softcon-vit-small-patch14-s1 | ViT-S/14 | S1 SAR | 2 | 384 | B2_vits14_softcon_enc.pth |
softcon-vit-base-patch14-s1 | ViT-B/14 | S1 SAR | 2 | 768 | B2_vitb14_softcon_enc.pth |
Usage
Processors default to do_resize: false. Pass patches at native (H, W, C); the processor rescales to [0, 1] without changing spatial size.
from transformers import pipeline
import numpy as np
REPO = "BiliSakura/SOFTCON-transformers"
SUBFOLDER = "softcon-vit-small-patch14-s2c"
pipe = pipeline(
task="softcon-feature-extraction",
model=REPO,
trust_remote_code=True,
model_kwargs={"subfolder": SUBFOLDER},
)
# S2-L1C: 13 bands at native resolution (e.g. 512×512)
image = np.random.randint(0, 255, (512, 512, 13), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape) # [1, 384]
Dense token features:
tokens = pipe(image, pool=False, return_tensors=True)
Opt in to resize (patch-14 models were pretrained on 224×224):
features = pipe(image, pool=True, return_tensors=True, image_processor_kwargs={"do_resize": True})
Load components directly:
from transformers import AutoModel, AutoImageProcessor
model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
Normalization
By default, the bundled image processor rescales inputs to [0, 1] by dividing by 255. SoftCon recommends mapping each channel to uint8 using per-channel mean/std from SSL4EO-S12 or the target dataset before inference. Enable do_normalize=True with per-channel image_mean and image_std on SoftConImageProcessor when using normalized inputs.
Dependencies
transformers,torch,torchvision,safetensorsopencv-python(multispectral resize with more than 4 channels)
Citation
@misc{wang2024multilabel,
title={Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining},
author={Wang, Yi and Albrecht, Conrad M and Zhu, Xiao Xiang},
journal={arXiv preprint arXiv:2405.20462},
year={2024}
}