ALHVR Reproduction and Class-Aware Threshold Extension — Checkpoints

Overview

This repository holds the trained checkpoints of a course project that reproduces ALHVR — Adaptive Learning of High-Value Regions for Semi-Supervised Medical Image Segmentation (Tao Lei, Ziyao Yang, Xingwu Wang, Yi Wang, Xuan Wang, Feiman Sun, Asoke K. Nandi) — on the ACDC cardiac MRI dataset with 10% labeled data, and extends it. The project:

• reproduces ALHVR, its two module ablations, and the U-Net and CPS baselines;
• evaluates the reproduction against the values reported in the paper;
• introduces and evaluates a class-aware confidence threshold γ_c as an alternative to ALHVR's single global threshold;
• reports additional sensitivity and capacity experiments: the threshold fraction β, the sharpening temperature T, network width, and a frozen DINOv2 encoder branch.

The repository contains 43 runs (15 for the reproduction, 28 for the extensions). Each run trained for 30,000 iterations at batch size 16. The training and evaluation code is public: github.com/fahimu10/prl-alhvr-reproduction.

These are research artifacts from a university project. They are not a clinical tool and must not be used for diagnosis.

Files

There is one directory per run, named <variant>_10pct_bs16_seed<seed>_<timestamp>/. Each contains:

The two-network methods (CPS, ALHVR and its variants) report network 1, as the reference implementation does.

Results

All values come from each run's results.json. The test set is 40 volumes (20 patients × 2 cardiac phases). Dice and Jaccard are in %. 95HD and ASD are computed with MedPy without voxel spacing, exactly as in the official evaluation script, so they are in voxel units. Reproduced values are mean ± standard deviation over seeds 1337, 2024 and 42.

Part 1: ALHVR reproduction

Values in parentheses (italic) are those reported in the ALHVR paper, which gives single values without seeds or variance.

Methods (paper Table 2, ACDC, 10% labeled)

Ablations (paper Table 5)

• ALHVR reproduces within ±5% of the paper on all four metrics, using the three-seed mean (Dice −0.9%, Jaccard −1.5%, 95HD −4.5%, ASD −3.3%).
• Every method is within ±5% on Dice and Jaccard. The method ordering (U-Net < CPS < ALHVR) and the ablation ordering (each module helps on its own, both together help most) also reproduce.
• Boundary metrics deviate more for the other methods. DTCT-only 95HD is +105% from the paper's value, driven by a few cases with stray components. CPS has lower 95HD and ASD than published (−37% and −30%), plausibly because the project's CPS is a construction rather than a port. The U-Net baseline's +23.6% 95HD lies within its own seed spread.

Part 2: extensions

Every row is full ALHVR with one change. Δ Dice is the paired difference in mean Dice from the ALHVR baseline (published β = 0.8), computed over the same 40 test volumes after averaging each variant's per-case values across its seeds; p is from the corresponding paired test. Eleven comparisons were made, so the Bonferroni-corrected threshold is p < 0.005; p-values in bold clear it. Rows with n = 1 are single-seed exploratory runs (seed 1337).

Jaccard, 95HD, ASD, per-class and per-case values for every run are in its results.json.

Key findings

1. Among the five β values tested, ALHVR's published β = 0.8 gave the lowest mean Dice. β = 0.9 improved Dice by +0.78 (p = 0.0001) and raised every class (RV +0.99, p = 0.0013; myocardium +0.59, p = 0.0004; LV +0.75, p = 0.015). It also had the smallest seed-to-seed spread of any three-seed variant (σ = 0.09) and the lowest mean 95HD and ASD. However, no 95HD comparison reaches p < 0.05 at three seeds. This finding concerns ALHVR's published configuration and is not a contribution of this project.
2. More capacity improved Dice, but no more than β = 0.9. Width 32 gave +0.86 Dice (p = 0.0005), but was statistically indistinguishable from β = 0.9 head to head (+0.078, p = 0.56). It also took roughly 1.7× the training time and had a higher mean 95HD (3.28 vs 1.93). The time ratio is indicative only, because the GPU type of each run was not recorded. Among the tested settings, β = 0.9 reached the same accuracy at lower cost.
3. For γ_c, the direction of the per-class adjustment mattered.
   • The quantile and offset formulations derive each class's threshold from that class's own confidences. Foreground classes are less confident than background, so their thresholds fell and fewer of their pixels reached ALHVR's two modules. At iteration 2000 the quantile formulation cut RV's low-confidence coverage from 58% to 13%. Both formulations reduced Dice by 2.8–3.5 points (one seed each).
   • The inverse formulation raises the threshold for under-confident classes. It gave +0.31 Dice over the baseline (p = 0.046, three seeds), which does not clear the Bonferroni threshold. RV Dice rose by 0.59, short of the +1.0 hypothesised in the Part 2 proposal.
   • The direction of the adjustment is clearly established (inverse vs quantile: +3.085 Dice, p < 0.0001). Its benefit over the global threshold is marginal.

Secondary results:

• Frozen DINOv2 features gave no overall gain (+0.28, p = 0.40). RV improved by 1.50 (p = 0.0053) while LV fell by 0.66 and 95HD worsened. The input normalisation was not matched to DINOv2's pretraining, so this result applies to DINOv2 as integrated here. Width 32 combined with DINOv2 (one seed) scored 88.95, below the baseline and below either change on its own.
• Sharpening: turning off DTCT's sharpening step (T = 1.0, one seed) changed Dice by −0.09 (p = 0.81). The linear warm-up gave +0.49 (p = 0.065), which is not significant.
• Boundary metrics: no Part 2 comparison of 95HD reaches p < 0.05 at three seeds, so 95HD and ASD differences are directional only.

Using the checkpoints

The model classes are in src/models/unet.py of the project repository.

import json
import torch
from src.models.unet import UNet_fea_aux   # run from the project repository's root

run = "alhvr_10pct_bs16_seed1337_<timestamp>"   # any ALHVR run directory
cfg = json.load(open(f"{run}/config.json"))
model = UNet_fea_aux(in_chns=1, class_num=4,
                     width=cfg.get("width", 16),
                     use_dino=bool(cfg.get("use_dino")),
                     dino_pretrained=False)
model.load_state_dict(torch.load(f"{run}/best_model1.pth", map_location="cpu"))
model.eval()
logits = model(x)[0]   # x: (N, 1, 256, 256); output 0 is the segmentation

• Architecture settings: read width and use_dino from the run's config.json, as above. Older runs omit these keys; their defaults are width 16 and no DINOv2.

• Output: UNet returns the segmentation logits directly.

• DINOv2 runs (*-dino*) embed the frozen DINOv2 weights, which is why their checkpoints are about 93 MB instead of about 7.3 MB. Loading them requires timm; dino_pretrained=False avoids downloading weights.

• Official ALHVR code: the 36 width-16 checkpoints without DINOv2 use the same module names as the official implementation, so they also load into its UNet and UNet_fea_aux (train_ALHVR/code/networks/unet.py in github.com/ziziyao/ALHVR).

• Test metrics: to reproduce them, run from the project repository's root with the prepared ACDC dataset:

  python -m src.evaluation.evaluate_acdc --root-path data/acdc_processed/data --checkpoint <run>/best_model1.pth --model unet_fea_aux
  

  Use --model unet for the U-Net baseline and CPS. Each slice is resized to 256 × 256 (nearest neighbour) and the prediction is resized back.

• Class indices: 0 background, 1 right ventricle (RV), 2 myocardium, 3 left ventricle (LV).

Training setup

• Data: ACDC. The 10% labeled subset is the first 136 training slices (7 patients), fixed rather than sampled. Train/validation/test splits come from the official ALHVR repository: 1,312 training slices, 20 validation volumes and 40 test volumes.
• Backbone: U-Net, 1.81 M parameters at base width 16. The two-network methods train two of them.
• Schedule: 30,000 iterations at batch size 16 (8 labeled + 8 unlabeled for the semi-supervised methods). SGD with learning rate 0.01, momentum 0.9, weight decay 1e-4, and no learning-rate schedule.
• Input: 256 × 256, per-volume min-max normalised to [0, 1].
• Hardware: RTX 3070 for the U-Net baseline and CPS; A100, RTX 3080 and V100 for ALHVR, the ablations and the Part 2 runs.
• Framework: PyTorch 2.6 (cluster module) for the cluster runs. The DINOv2 runs used a separate PyTorch 2.14 installation with timm.

Two reference quirks kept deliberately

Both are faithful ports of the official implementation, not bugs introduced here. Reproducing these numbers requires them.

1. DiceLoss omits the factor of 2 in its numerator, so a perfect prediction floors the term at 0.5 rather than 0.
2. compute_prototype_loss broadcasts its mask to (N, N, H, W), inflating the CG-CPCL cross-entropy term by roughly a factor of N. This entangles the loss with the batch size, which is one reason batch size 16 is fixed across every run.

Limitations

• Scope: one dataset (ACDC), one label fraction (10%), and 2D slice-wise segmentation.
• Single-seed rows: width 32 + DINOv2, T = 1.0 and the two failed γ_c formulations are single-seed; their numbers are indicative, not established. Three conclusions drawn from single-seed runs earlier in the project reversed at three seeds, which is why every claim above rests on three seeds.
• Seed spread: the labeled subset is fixed, so the reported spread covers training stochasticity only.
• Boundary metrics: 95HD and ASD are unstable at this sample size. Their seed-to-seed standard deviation ranges from 4.5% of the mean (full ALHVR) to 58% (DTCT only), so prefer Dice and Jaccard for comparisons.
• Reproducibility: boundary metrics are not bit-reproducible, even at a fixed seed, because a CUDA upsampling kernel is non-deterministic.
• Clinical use: no clinical validation of any kind.

License

These checkpoints, like the rest of this project's own work, are released under CC BY-NC 4.0 (Attribution-NonCommercial).

• Ported ALHVR code: the checkpoints were trained with code partly ported or adapted from the official ALHVR implementation, which declared no license when it was cloned for this project. The CC BY-NC 4.0 license does not extend to that code, which remains subject to whatever rights the ALHVR authors hold. Each ported file names its upstream source in its module docstring.
• DINOv2 weights: runs with -dino in the name embed frozen vit_small_patch14_dinov2 weights obtained via timm. Those weights remain under their own license.
• ACDC data: the dataset is not redistributed here. It must be obtained from the MICCAI 2017 challenge organisers.

See the project's LICENSE for details.

Acknowledgements

This project builds on the ALHVR paper and its official implementation:

• Tao Lei, Ziyao Yang, Xingwu Wang, Yi Wang, Xuan Wang, Feiman Sun and Asoke K. Nandi. Adaptive Learning of High-Value Regions for Semi-Supervised Medical Image Segmentation.
• Official implementation: github.com/ziziyao/ALHVR. It is the sole reference for every implementation detail reproduced here.

The official implementation states that its code is adapted from CCT, MC-Net, UPCoL and SSL4MIS, and its U-Net architecture is attributed to PyMIC. See the ALHVR repository's README for the prior work its authors credit.