ViT-Large — Skin Cancer Classification (HAM10000)
Fine-tuned google/vit-large-patch16-224 on the HAM10000 dermoscopy dataset to classify 7 types of skin lesions. Trained on a free Google Colab T4 GPU in 37 minutes using HuggingFace Trainer with FP16 mixed precision.
This was a personal project to learn medical image fine-tuning — not a clinical tool.
How to use
from transformers import pipeline
classifier = pipeline(
"image-classification",
model="Kuldeepmishra3/vit-large-skin-cancer-ham10000",
)
results = classifier("skin_image.jpg", top_k=3)
for r in results:
print(f"{r['label']}: {r['score']:.2%}")
Or with manual preprocessing:
from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch
model = ViTForImageClassification.from_pretrained("Kuldeepmishra3/vit-large-skin-cancer-ham10000")
processor = ViTImageProcessor.from_pretrained("Kuldeepmishra3/vit-large-skin-cancer-ham10000")
image = Image.open("skin_image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
predicted_class = logits.argmax(-1).item()
print(model.config.id2label[predicted_class])
Classes
| Label | Full name |
|---|---|
| akiec | Actinic Keratoses |
| bcc | Basal Cell Carcinoma |
| bkl | Benign Keratosis |
| df | Dermatofibroma |
| mel | Melanoma |
| nv | Melanocytic Nevi |
| vasc | Vascular Lesions |
Results
Evaluated on the HAM10000 validation split (2,492 images):
| Metric | Score |
|---|---|
| Accuracy | 92.74% |
| Weighted F1 | 92.60% |
Per-class F1:
| Class | F1 |
|---|---|
| vasc | 97.14% |
| nv | 96.54% |
| df | 92.86% |
| bcc | 92.24% |
| bkl | 84.37% |
| akiec | 82.50% |
| mel | 79.78% |
Melanoma was the hardest — it frequently gets confused with Melanocytic Nevi, which is a known challenge in dermoscopy even for trained dermatologists.
Training details
- Base model: google/vit-large-patch16-224 (303M params)
- Dataset: HAM10000 via marmal88/skin_cancer on HuggingFace
- Epochs: 5 (best checkpoint at epoch 4)
- Batch size: 16
- Learning rate: 2e-5 with cosine schedule
- Warmup steps: 300
- Precision: FP16
- GPU: Tesla T4 (Google Colab free tier)
- Training time: ~37 minutes
Augmentations used during training: random horizontal/vertical flip, rotation up to 30°, color jitter. No augmentation at validation time.
Limitations
- Not a clinical diagnostic tool — do not use for real medical decisions
- Performance drops on minority classes (akiec, df, vasc) due to class imbalance in HAM10000
- Only trained on dermoscopy images — won't work well on regular phone camera skin photos
- Dataset is mostly from European populations which may affect generalization
Dataset
HAM10000 (Human Against Machine with 10000 training images) is a large collection of multi-source dermatoscopic images of common pigmented skin lesions. The dataset has significant class imbalance — Melanocytic Nevi makes up around 67% of all samples.