LANTERN — Aβ plaque segmentation in 3D light-sheet microscopy (MAPT model C: 3-class, asymmetric CE, topology-refined)

5-fold ensemble of 3D U-Nets that segment amyloid-β plaques in cleared whole mouse brain imaged by light-sheet microscopy, fine-tuned on the MAPT/APOE cohort from apooladi/lantern-ki3-abeta. Each fold is the same frozen self-supervised ResEncL encoder with a fine-tuned decoder. This is one of three sibling releases (lantern-mapt_a/b/c-abeta) trained on identical data with different label handling; they are being compared visually on held-out 512³ crops before one is chosen for the whole-dataset run.

LANTERN = Light-sheet Automask Network for Transferable Embedding Representations.

What this is

Recipe

3-class head (margin 4); cross-entropy on background voxels relaxed to -log(p0 + 0.5*p2) so calling background 'FP' is cheap and calling it 'plaque' is not (--bg-fp-slack 0.5), Dice on both foreground classes; then a 12-epoch refinement from the best-validation weights (lr 1e-4) adding a cubical persistent-homology penalty on the product of plaque and FP probabilities (target: zero components, i.e. plaque never touches FP; weight 0.04) and an anchor to the pre-refinement prediction (weight 1.0). Final weights of the refinement.

Intended use

Detecting and delineating Aβ plaques in the MAPT/APOE mouse cohort (Abeta channel, 4x, near-iso grid above). Research model for plaque-burden quantification, not a diagnostic tool. Class 2 marks bright non-plaque structures the annotators labelled as false positives (vessels, tissue edges, ventricle walls); it is a training device and is not a validated output. Only the plaque channel is scored.

Usage

Preprocessing must match exactly — the model is scale- and normalisation-sensitive (see Limitations).

import numpy as np, torch

def normalize(vol):                      # per-tile, exactly as in training
    lo, hi = np.percentile(vol, [0.5, 99.5])
    v = np.clip(vol, lo, hi).astype(np.float32)
    s = float(v.std())
    return (v - float(v.mean())) / (s if s > 1e-8 else 1.0)

# `ResEncLUNet` comes from the LANTERN repo (lantern/models/unet_seg.py)
nets = []
for k in range(5):
    ck = torch.load(f"fold{k}/seg_model.pt", map_location="cpu", weights_only=False)
    n = ResEncLUNet(num_classes=3, num_input_channels=1)
    n.load_state_dict(ck["state_dict"])
    nets.append(n.eval().cuda())

x = torch.from_numpy(normalize(tile))[None, None].cuda()   # tile: float32 (128,128,128)
votes = torch.zeros((128, 128, 128), dtype=torch.uint8, device="cuda")
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
    for n in nets:
        votes += (torch.softmax(n(x).float(), 1)[0, 1] >= 0.5).to(torch.uint8)   # plaque channel
mask = (votes >= 3)

For volumes larger than 128³, tile at stride 64 (50% overlap) and combine overlapping tiles with a max over votes. Recommended post-processing: drop predicted plaque components under 8 near-iso voxels (+0.06 object F1 on held-out data).

Training data

40 hand-annotated 128³ patches from 18 MAPT/APOE mice (v7/mapt/mapt_set/v3), Abeta channel: 470 plaque objects, 16 patches carrying annotated false positives (label 2), 8 patches with no plaque. Every object was hand-decided (two annotators, 2026-09-09/10); footprints were then re-drawn mechanically at one fixed threshold in normalised intensity (z > 3.0 within 2 voxels of the drawn object) so that plaque extent is consistent across patches. 32 train / 8 val per fold, random patch-level split, seed 1234.

Augmentation: axis flips, one 90° rotation, mild gamma and Gaussian noise. AdamW, lr 3e-4, 5-epoch warm-up then cosine, 150 epochs, batch 2 × 4 accumulation, bf16-mixed.

Evaluation

Held-out 5-fold, object F1 at IoU ≥ 0.5 on 26-connected components, micro-averaged over 470 plaques.

For reference, the pass-2 recipe scored 0.129 on the same patches before the extent harmonisation; the ki3 model before MAPT fine-tuning is what binary_mask_neariso_512 shows in the QC bank. The three sibling models are within 0.03 of each other, inside fold noise.

Limitations

• Scale and normalisation sensitivity as in the ki3 card: resample to 4.0 × 3.25 × 3.25 µm and normalise per tile on raw intensities.
• Optimistic validation. Patch-level split, not subject-grouped; expect worse on an unseen animal.
• Small training set. 40 patches; every fold overfits after its validation peak, which is why the refinement is started from best-validation weights.
• Ground truth is one convention. Extent was harmonised mechanically; the 42 cases where adjacent cores merged into one object follow the rule, not an annotator.
• FP head bistability. The class-2 head is active in only some folds; a majority vote on class 2 returns nothing, so treat class 2 as advisory (any fold) or ignore it.
• Not loadable by spimquant's LANTERN plugin as-is (it assumes a 2-class head); the plaque channel is softmax[:, 1] as before.
• Not validated for absolute burden. Detection metrics on 128³ patches only.

Files

fold0..fold4/seg_model.pt   torch checkpoint: state_dict (final weights), config, provenance keys
config.json                 architecture + preprocessing contract

Citation

Code: https://github.com/Arshya-Guru/LANTERN

Released for non-commercial research use (CC BY-NC 4.0). The underlying imaging data is not included.