ConvNeXt V2 + Swin Transformer + Cross-Attention Fusion — Kvasir Dataset v2 (8-class GI Endoscopy)
8-class classifier (dyed-lifted-polyps, dyed-resection-margins, esophagitis, normal-cecum, normal-pylorus, normal-z-line, polyps, ulcerative-colitis) for GI endoscopy frames, trained on Kvasir-Dataset-v2 with stratified group 10-fold cross-validation.
Architecture
- ConvNeXt V2 (tiny) backbone — local/texture features
- Swin Transformer (tiny) backbone — global/contextual features
- Bidirectional cross-attention fusion (2 layers) between both token streams
- MLP classification head
10-Fold Cross-Validation Results (mean ± std)
| Metric | Value |
|---|---|
| Accuracy | 0.9514 ± 0.0057 |
| Precision (macro) | 0.9520 ± 0.0054 |
| Recall (macro) | 0.9514 ± 0.0057 |
| F1 (macro) | 0.9513 ± 0.0057 |
| ROC-AUC (macro) | 0.9959 ± 0.0014 |
Usage
from huggingface_hub import hf_hub_download
# see inference.py in this repo for load_model() / predict()
Explainability
Grad-CAM visualizations for both the ConvNeXt and Swin streams are included in the original training notebook to support clinical interpretability.