LANTERN — Aβ plaque segmentation in 3D light-sheet microscopy (MAPT model A: 3-class, plain DiceCE, GT floor 8)
5-fold ensemble of 3D U-Nets that segment amyloid-β plaques in cleared whole mouse brain imaged by
light-sheet microscopy, fine-tuned on the MAPT/APOE cohort from
apooladi/lantern-ki3-abeta. Each fold is the
same frozen self-supervised ResEncL encoder with a fine-tuned decoder. This is one of three sibling
releases (lantern-mapt_a/b/c-abeta) trained on identical data with different label handling; they
are being compared visually on held-out 512³ crops before one is chosen for the whole-dataset run.
LANTERN = Light-sheet Automask Network for Transferable Embedding Representations.
What this is
| Architecture | ResEncLUNet — ResEncL encoder (6 stages, features [32,64,128,256,320,320]) + UNetDecoder |
| Parameters | 102.3 M total, 12.1 M trainable (encoder frozen) |
| Encoder init | nnBYOL3D self-supervised pretraining on ~50k near-iso-4 light-sheet patches (unchanged since) |
| Decoder init | lantern-ki3-abeta, fold-matched |
| Input | 1 channel, 128³ voxels, near-iso grid (4.0 × 3.25 × 3.25 µm) |
| Output | 3 classes (background, plaque, annotated false positive) |
| Weights | best |
| Folds | 5, patch-level 5-fold CV — use all 5 and vote |
Recipe
3-class head (background / plaque / annotated false positive) widened from the ki3 2-class head (bias margin 4, argmax preserved at init); plain DiceCE (CE over 3 classes + batch-Dice on both foreground classes); ground-truth components under 8 voxels removed before training and validation (--gt-min-cc 8); best-validation checkpoint.
Intended use
Detecting and delineating Aβ plaques in the MAPT/APOE mouse cohort (Abeta channel, 4x, near-iso grid above). Research model for plaque-burden quantification, not a diagnostic tool. Class 2 marks bright non-plaque structures the annotators labelled as false positives (vessels, tissue edges, ventricle walls); it is a training device and is not a validated output. Only the plaque channel is scored.
Usage
Preprocessing must match exactly — the model is scale- and normalisation-sensitive (see Limitations).
import numpy as np, torch
def normalize(vol): # per-tile, exactly as in training
lo, hi = np.percentile(vol, [0.5, 99.5])
v = np.clip(vol, lo, hi).astype(np.float32)
s = float(v.std())
return (v - float(v.mean())) / (s if s > 1e-8 else 1.0)
# `ResEncLUNet` comes from the LANTERN repo (lantern/models/unet_seg.py)
nets = []
for k in range(5):
ck = torch.load(f"fold{k}/seg_model.pt", map_location="cpu", weights_only=False)
n = ResEncLUNet(num_classes=3, num_input_channels=1)
n.load_state_dict(ck["state_dict"])
nets.append(n.eval().cuda())
x = torch.from_numpy(normalize(tile))[None, None].cuda() # tile: float32 (128,128,128)
votes = torch.zeros((128, 128, 128), dtype=torch.uint8, device="cuda")
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
for n in nets:
votes += (torch.softmax(n(x).float(), 1)[0, 1] >= 0.5).to(torch.uint8) # plaque channel
mask = (votes >= 3)
For volumes larger than 128³, tile at stride 64 (50% overlap) and combine overlapping tiles with a
max over votes. Recommended post-processing: drop predicted plaque components under 8 near-iso voxels
(+0.06 object F1 on held-out data).
Training data
40 hand-annotated 128³ patches from 18 MAPT/APOE mice (v7/mapt/mapt_set/v3), Abeta channel:
470 plaque objects, 16 patches carrying annotated false positives (label 2), 8 patches with no
plaque. Every object was hand-decided (two annotators, 2026-09-09/10); footprints were then
re-drawn mechanically at one fixed threshold in normalised intensity (z > 3.0 within 2 voxels of the
drawn object) so that plaque extent is consistent across patches. 32 train / 8 val per fold, random
patch-level split, seed 1234.
Augmentation: axis flips, one 90° rotation, mild gamma and Gaussian noise. AdamW, lr 3e-4, 5-epoch warm-up then cosine, 150 epochs, batch 2 × 4 accumulation, bf16-mixed.
Evaluation
Held-out 5-fold, object F1 at IoU ≥ 0.5 on 26-connected components, micro-averaged over 470 plaques.
| as predicted | + drop components < 8 voxels | |
|---|---|---|
| object F1 | 0.364 | 0.428 |
| precision / recall | 0.376 / 0.353 | |
| false positives per patch | 6.9 | 3.2 |
| detection F1 (any-overlap match) | 0.544 | 0.564 |
| voxel Dice | 0.330 |
For reference, the pass-2 recipe scored 0.129 on the same patches before the extent
harmonisation; the ki3 model before MAPT fine-tuning is what binary_mask_neariso_512 shows in the
QC bank. The three sibling models are within 0.03 of each other, inside fold noise.
Limitations
- Scale and normalisation sensitivity as in the ki3 card: resample to 4.0 × 3.25 × 3.25 µm and normalise per tile on raw intensities.
- Optimistic validation. Patch-level split, not subject-grouped; expect worse on an unseen animal.
- Small training set. 40 patches; every fold overfits after its validation peak, which is why best-validation weights are shipped.
- Ground truth is one convention. Extent was harmonised mechanically; the 42 cases where adjacent cores merged into one object follow the rule, not an annotator.
- FP head bistability. The class-2 head is active in only some folds; a majority vote on class 2 returns nothing, so treat class 2 as advisory (any fold) or ignore it.
- Not loadable by spimquant's LANTERN plugin as-is (it assumes a 2-class head); the plaque channel is
softmax[:, 1]as before. - Not validated for absolute burden. Detection metrics on 128³ patches only.
Files
fold0..fold4/seg_model.pt torch checkpoint: state_dict (best weights), config, provenance keys
config.json architecture + preprocessing contract
Citation
Code: https://github.com/Arshya-Guru/LANTERN
Released for non-commercial research use (CC BY-NC 4.0). The underlying imaging data is not included.