VideoMAE Tennis Shot Type Classifier
Fine-tuned VideoMAE-base (86M params) for 5-class tennis shot classification from 16-frame video clips.
Classes
| Label | ID |
|---|---|
| Topspin | 0 |
| Slice | 1 |
| Volley | 2 |
| Flat | 3 |
| Smash | 4 |
Training Details
- Dataset: 2,436 annotated hit events from TenniSet (videos V006-V010)
- Input: 16 frames x 224x224, sampled around each hit timestamp
- Accuracy: 88-90% on held-out test split
- Hardware: Apple M4 Pro (48GB, MPS)
- Framework: HuggingFace Transformers
Usage
from transformers import VideoMAEImageProcessor, VideoMAEForVideoClassification
processor = VideoMAEImageProcessor.from_pretrained("vishnushenoy09/videomae-tennis-shottype")
model = VideoMAEForVideoClassification.from_pretrained("vishnushenoy09/videomae-tennis-shottype")
Credits
- VideoMAE by Zhan Tong, Yibing Song, Jue Wang, Limin Wang (MCG-NJU). Paper: VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training (NeurIPS 2022).
- TenniSet dataset by Kacper Wozniak. Used for training data annotation.
- Ball tracking and bounce detection adapted from yastrebksv/TrackNet and yastrebksv/TennisProject.
Part of
This model is part of the Sports Commentator from Video project — an automated tennis commentary pipeline using TrackNet, CatBoost, VideoMAE, Qwen2.5, RF-DETR, and F5-TTS.