Multi-Species Crab Detection with RF-DETR Medium
Automated object detection and multi-class identification system for three crab species: European Green Crab, Rock Crab, and Jonah Crab. Trained using the RFDETRMedium transformer architecture on a merged multi-source dataset on Kaggle.
Model Performance & Evaluation
Benchmark Summary
| Checkpoint | Split | mAP@50 | mAP@50-95 | mAP@75 |
|---|---|---|---|---|
| checkpoint_best_total | Valid | 0.9922 | 0.8600 | 0.9348 |
| checkpoint_best_total | Test | 0.9834 | 0.8147 | 0.8910 |
| last_ema | Valid | 0.9877 | 0.8558 | 0.9322 |
| last_ema | Test | 0.9786 | 0.8178 | 0.8879 |
Training & Validation Progression
The model was trained for 15 epochs, reaching optimal bounding box precision at epoch 5 before early convergence.
Confusion Matrices (Raw Counts)
Evaluated with confidence threshold ≥0.50 and IoU threshold ≥0.50. The matrices display true positives along the diagonal, cross-species confusion in off-diagonal cells, background false positives (FP), and missed crabs (FN).
(Note: Insert images of your confusion matrices here for the Validation Split (578 Images) and Test Split (289 Images))
Dataset Overview
The dataset was constructed by aggregating five open-source datasets into a consolidated COCO-format collection.
- Base Dataset Images: 2,889 images with 6,630 annotated objects (2.3 annotations/image average).
- Base Object Distribution: 2,405 single-object images, 120 images with 2–5 objects, and 447 multi-object scenes.
- Dimensions: 81.8% medium-resolution images, median resolution 386×342.
Class Distribution (Base Split)
| Class Name | Train Annotations | Valid Annotations | Test Annotations | Total Count |
|---|---|---|---|---|
| European Green Crab | 1,780 | 500 | 277 | 2,557 |
| Rock Crab | 1,538 | 417 | 249 | 2,204 |
| Jonah Crab | 1,285 | 358 | 226 | 1,869 |
Preprocessing & Data Augmentation
To avoid overfitting and improve generalization across unseen marine environments, a 5× training augmentation pipeline was applied:
- Preprocessing: Auto-orientation and white-padded resizing to 576×576 pixels.
- Rotations: Random rotation between -15° and +15°.
- Photometric Distortions: Brightness (±15%) and exposure (±15%) adjustments.
- Sensor Noise & Artifacts: Up to 2% random pixel noise and 20px directional motion blur (45° angle).
Augmented Dataset Distribution:
- Train Set: 10,110 images (92%)
- Validation Set: 578 images (5%)
- Test Set: 289 images (3%)
- Total Volume: 10,977 images
Training Configuration
| Setting | Value |
|---|---|
| Base Model | RFDETRMedium with dinov2_windowed_small backbone |
| Input Resolution | 576×576 pixels |
| Batch Strategy | Batch size 4 with 4 grad accumulation steps (effective 16) |
| Optimizer | AdamW (lr=1e-4, lr_encoder=1.5e-4, weight_decay=1e-4) |
| Early Stopping | Patience 10 epochs (min delta 0.001) |
| Compute | Kaggle GPU environment (~12 hours total) |
Usage
Install Dependencies
pip install rfdetr huggingface_hub
Load Model from Hugging Face
from huggingface_hub import hf_hub_download
import rfdetr
weights = hf_hub_download(
repo_id="hojayfa-sir/european-green-crab-detection",
filename="european-green-crab-detection.pth"
)
model = rfdetr.RFDETRMedium(pretrain_weights=weights)
Run Inference
detections = model.predict("image.jpg", threshold=0.50)
Citation & Attribution
Author: Abu Hojayfa
If you use this model in your research or MATE ROV application, please consider citing it:
@misc{crab_rfdetr_model,
title={Multi-Species Crab Detection with RF-DETR Medium},
author={Abu Hojayfa},
year={2026},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/hojayfa-sir/european-green-crab-detection}}
}