rtdetr-sportsmot — RT-DETRv2 (r50vd) fine-tuned on SportsMOT for player detection

Single-class (player) detector fine-tuned from PekingU/rtdetr_v2_r50vd on the annotated train sequences of Lekim89/sportsmot (basketball, soccer, and volleyball broadcast clips in MOTChallenge format). Intended as the detection stage of a tracking-by-detection pipeline (e.g. with ByteTrack via the supervision library) for basketball player tracking.

Try it in one click

• Live demo Space — a Gradio app on ZeroGPU: drop a sports frame, tune the confidence threshold, get annotated player boxes.
• Notebook (notebook.ipynb): open in Colab or Kaggle, or via the "Use this model" button above — install → load → detect → visualize, no token needed, CPU or GPU.

Validation (held-out basketball sequence v_-6Os86HzwCs_c009)

Usage

Note: this is a vision model — its preprocessor is an AutoImageProcessor (there is no tokenizer). Ignore the auto-generated "Load model directly" snippet in the Use this model button if it shows AutoTokenizer — that call will error. Use either option below.

Simplest — pipeline:

from transformers import pipeline

pipe = pipeline("object-detection", model="smallTech/rtdetr-sportsmot")
results = pipe("frame.jpg", threshold=0.5)   # [{'score', 'label': 'player', 'box'}, ...]

Manual — processor + model (full control over pre/post-processing):

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForObjectDetection

processor = AutoImageProcessor.from_pretrained("smallTech/rtdetr-sportsmot")
model = AutoModelForObjectDetection.from_pretrained("smallTech/rtdetr-sportsmot")

image = Image.open("frame.jpg")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
results = processor.post_process_object_detection(
    outputs, target_sizes=torch.tensor([image.size[::-1]]), threshold=0.5
)

Training details

• Approach: RT-DETRv2's COCO detection head is replaced with a one-class (player) head while reusing the pretrained backbone.
• Data: the annotated train/ split of SportsMOT — 44 of its 45 sequences for training (14,049 frames at stride 2, frames without boxes skipped), with one basketball sequence (v_-6Os86HzwCs_c009, 100 frames at stride 5) held out for validation. Annotations are MOTChallenge gt.txt boxes; conf==0 ignore-boxes are skipped, track ids are unused for detection. The val/ and test/ splits were never touched by training — they are the benchmark sets below.
• Training ran entirely on Kaggle's free tier (single T4, fp16): batch 4 with 4-step gradient accumulation (effective 16), 20 epochs at 640×640, lr 5e-5 cosine with 300-step warmup, augmentation (hflip, color jitter, gaussian noise).
• Model selection: checkpoint with the best held-out eval_loss — epoch 11 of 20 (later epochs overfit).

Benchmark — unseen val split (45 sequences)

Evaluated on every annotated frame of the SportsMOT val split — 26,970 frames / 295,573 ground-truth boxes across 45 sequences (basketball, soccer, volleyball) never used in training (training consumed only the train/ split; test/ has no public ground truth). Frame stride 1; same metric implementation as the training-time validation (torchmetrics COCO mAP, detection threshold 0.01).

Latency (single image, batch 1, fp32, Tesla T4, torch 2.10.0+cu128): mean 61.6 ms · p50 61.4 ms · p90 62.8 ms · p99 66.3 ms → 16.2 fps. Model load: 5.4 s.

Benchmark — test split (performance only)

Measured on the full SportsMOT test split (150 sequences, 94,835 frames), staged as two mutually exclusive halves to fit Kaggle's kernel-output cap. The test split has no public ground truth (withheld for the SportsMOT challenge server), so no accuracy metrics are possible here — see the val-split benchmark above for those.

Environment & method — the assumptions these numbers were measured under:

Pipeline & reproducibility

This model was trained, evaluated, and benchmarked end-to-end on Kaggle's free tier, orchestrated from the HuggingFace workspace repository (staging → smoketest → run for each stage). The benchmark sections of this card are written by the evaluation/testing kernels themselves and update in place on re-runs.