DIR CRC Tissue-Compartment Segmentation Model (SegFormer)

Semantic segmentation of tissue compartments in H&E-stained colorectal cancer (CRC) whole-slide images. Part of the Digital Immune Reporter (DIR) pipeline.

• Architecture: SegFormer (MiT-B3 backbone), via MMSegmentation
• Task: 4-class tissue-compartment semantic segmentation on H&E tiles
• Classes: Tumor, Stroma, Necrosis, Other (plus an internal background/unlabeled class at index 0)

Files

• iter_40000.pth — model checkpoint (MMEngine format; includes meta.dataset_meta with the class-name list used at training time)
• segformer_b3_40k_2xb4_tcgacrc_tissue_augment.py — full MMSegmentation training config

Usage

from mmseg.apis import init_model, inference_model

model = init_model(
    "segformer_b3_40k_2xb4_tcgacrc_tissue_augment.py",
    "iter_40000.pth",
    device="cuda:0",
)
result = inference_model(model, "path/to/tile.png")

model.dataset_meta["classes"] gives the ordered class-name tuple corresponding to each predicted pixel value.

Training data

Trained on TCGA-CRC ROIs, using the same slide-level train/test split as the morphology-based cell-type model: 469 training (TCGA-train) and 109 held-out test (TCGA-test) ROIs. Tissue-compartment labels (tumor epithelium, tumor stroma, necrosis, other tissue) were generated using QuPath super-pixel classifiers. Images were processed at 0.25 mpp with 1024x1024 px patches (256x256 um) — larger than the cell-type model's patches, to capture enough tissue context for compartment-level segmentation.

Fifteen semantic-segmentation architectures were benchmarked over 40,000 iterations each: DeepLabV3+ (ResNet-50/101), FCN (ResNet-50/101), Swin-Tiny/Small/Base, ViT-Base/16, UperNet with a UNI adapter, and SegFormer B0-B5, evaluated on TCGA-test. SegFormer-B3 was selected as the final architecture, then retrained for 40,000 iterations (MMSegmentation v1.2.2) — this checkpoint.

Training used AdamW (lr 6e-5, weight decay 0.01, beta=(0.9, 0.999)) with layer-wise LR scaling (norm layers: decay multiplier 0.0; segmentation head: LR multiplier 10.0), a 400-iteration linear warm-up, and polynomial decay to zero over the remaining 39,600 iterations. Data augmentation included random sized cropping (scale 512-1024 px, p=0.5), random tone curve (scale 0.4) and gamma (40-60) adjustment, color jitter (hue +/-0.1, saturation +/-0.5), and random horizontal flip (p=0.5).

Per-class IoU/Dice for tumor epithelium and tumor stroma (the two compartments used downstream in spatial TIME analysis) were the primary evaluation metrics. Necrosis and background/other are known to underperform the other two classes — necrosis is under-represented in the training data, and QuPath-derived background labels were inconsistently distinguished from other, causing frequent confusion between the two.

License

CC-BY-NC-4.0 — research/non-commercial use only.