CV Multitask Pipeline
Multi-task computer vision model performing simultaneous object classification and bounding box detection on PASCAL VOC 2012 (20 classes).
Model Details
- Architecture: Frozen EfficientNet-B0 backbone + custom classification head + custom detection head
- Backbone: ImageNet-pretrained EfficientNet-B0 (frozen, 4M params)
- Classification head: 667K trainable params, outputs 20-class logits
- Detection head: 329K trainable params, outputs normalized bbox coordinates
Training
- Dataset: PASCAL VOC 2012 (5,717 train / 5,823 val images)
- Epochs: 30 (best checkpoint at epoch 5)
- Loss: Weighted multi-task loss (CrossEntropy + SmoothL1, λ_cls=1.0, λ_det=5.0)
- Hardware: Google Colab T4 GPU
Results
| Metric | Value |
|---|---|
| Top-1 Accuracy | 79.9% |
| Mean IoU | 0.468 |
| IoU@0.5 Accuracy | 49.2% |
Usage
import torch
from huggingface_hub import hf_hub_download
checkpoint_path = hf_hub_download(
repo_id="OmUniyal/cv-multitask-pipeline",
filename="best_model.pt"
)
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
Full model code and inference pipeline: GitHub repo
Limitations
- Closed-set classification — no "none of the above" option for out-of-distribution images
- Overfitting observed after epoch 5
- Single bbox regression — doesn't handle multiple objects per image