Retinal Disease Ensemble (ODIR-5K)
Multi-label classifier for the ODIR-5K fundus dataset. Takes paired left/right eye images and predicts 8 ocular conditions. Dual-backbone ensemble (EfficientNet-B4 + Inception-ResNet-v2), per-backbone input normalization, CLAHE preprocessing, per-class thresholds, and 4-view test-time augmentation.
Not a medical device. This is a portfolio project. It has not been clinically validated and must not be used to make or inform a diagnosis. Trained on a single dataset (ODIR-5K, ~3,500 patients); performance on images from other cameras, sites, or populations is unmeasured and likely worse.
The 8 classes: N (Normal), D (Diabetes), G (Glaucoma), C (Cataract), A (AMD), H (Hypertension), M (Myopia), O (Other).
Results
macro-AUC 0.888 (ensemble + TTA) on an 80/20 patient-level val split.
| Model | macro-AUC |
|---|---|
| EfficientNet-B4 | 0.870 |
| Inception-ResNet-v2 | 0.885 |
| Ensemble + TTA | 0.888 |
Per-class metrics at tuned thresholds:
| Class | Condition | AUC | F1 | Precision | Recall |
|---|---|---|---|---|---|
| N | Normal | 0.821 | 0.659 | 0.595 | 0.739 |
| D | Diabetes | 0.860 | 0.717 | 0.764 | 0.674 |
| G | Glaucoma | 0.959 | 0.649 | 0.667 | 0.633 |
| C | Cataract | 0.980 | 0.857 | 0.852 | 0.862 |
| A | AMD | 0.938 | 0.659 | 0.635 | 0.684 |
| H | Hypertension | 0.816 | 0.306 | 0.220 | 0.500 |
| M | Myopia | 0.996 | 0.889 | 0.889 | 0.889 |
| O | Other | 0.737 | 0.518 | 0.491 | 0.548 |
Myopia, Cataract, and Glaucoma are strong (distinctive signatures, clean labels). Hypertension is the weak class across the board: subtle signs, ~5% prevalence, fewer than 200 positive training examples. Other is a noisy catch-all whose ceiling is a labeling problem, not a modeling one.
Usage
The model is a custom ensemble, not a transformers architecture, so it loads through a small helper shipped in the repo (modeling_retinal.py) rather than AutoModel.
Requirements: torch, timm, opencv-python-headless, pillow, numpy, huggingface_hub, safetensors.
import importlib.util
from huggingface_hub import hf_hub_download
from PIL import Image
REPO = "Mjolnirslams/retinal-disease-ensemble"
code = hf_hub_download(REPO, "modeling_retinal.py")
spec = importlib.util.spec_from_file_location("modeling_retinal", code)
modeling = importlib.util.module_from_spec(spec)
spec.loader.exec_module(modeling)
model = modeling.load_ensemble_from_hub(REPO, device="cpu")
left = Image.open("left_fundus.jpg")
right = Image.open("right_fundus.jpg")
probabilities, predictions = model.predict(left, right, tta=True)
print(probabilities) # {"N": 0.87, "D": 0.04, ...}
print(predictions) # {"N": True, "D": False, ...} thresholded per class
predict runs the full preprocessing pipeline (CLAHE on the L channel, resize to 448, per-backbone normalization) and applies the tuned per-class thresholds stored in config.json. Pass tta=False for a single view (~4x faster, slightly lower accuracy).
Training
Two backbones trained independently, then averaged at inference.
- Framing: multi-label (sigmoid +
BCEWithLogitsLoss), not softmax. Patients present with co-occurring conditions, which softmax actively penalizes. - Dual-eye fusion: both eyes pass through a shared backbone in one forward pass; features are concatenated at the head so the model sees inter-eye asymmetry.
- Phase 1: head only, backbone frozen, 5 epochs, LR 1e-3.
- Phase 2: full fine-tune, up to 25 epochs, LR 1e-4, cosine annealing, early stopping on val AUC (patience 7).
- Optimizer: AdamW, weight decay 1e-3.
- Augmentation: RandomResizedCrop(448, 0.8-1.0), H/V flip, rotation 15 deg, ColorJitter.
- Class imbalance: per-class
pos_weight = neg/pospassed to the loss. - Thresholds: tuned per class on val to maximize F1, not a flat 0.5.
- Preprocessing: CLAHE (clipLimit 2.0, 8x8 tiles) on the L channel of LAB, applied on train and val.
- Normalization: EfficientNet-B4 uses ImageNet stats; Inception-ResNet-v2 uses [-1, 1] (0.5 mean/std), matching its pretraining.
Caveats
- Hypertension is a data-volume problem (AUC 0.816, F1 0.306). The model ranks H cases reasonably but binary predictions are unreliable. No architecture change fixes ~200 positive examples.
- "Other" is a labeling problem. O is a catch-all for conditions outside the first seven classes. Its ceiling is label noise.
- No demographic stratification. ODIR-5K lacks consistent demographic metadata, so per-subgroup performance (age, sex, ethnicity) is not evaluated.
- Single-source training. ODIR-5K only. The ~0.93 published range comes from teams pretraining on EyePACS/APTOS/MESSIDOR before ODIR-5K; that gap is a data problem, not an architecture one.
License and attribution
MIT.
Dataset: ODIR-5K (Ocular Disease Intelligent Recognition), available on Kaggle. Backbones are ImageNet-pretrained weights from timm.