YOLOFT for XS-VID v2
Task-specific YOLOFT-L checkpoints for XS-VID: A Large-Scale Benchmark for Small Object Detection and Tracking in Videos, accepted by TPAMI in 2026. DOI: 10.1109/TPAMI.2026.3741044.
- Project: https://gjhhust.github.io/XS-VID/
- Runtime, environment and all task commands: https://github.com/gjhhust/YOLOFT
- Dataset: https://huggingface.co/datasets/lanlanlan23/XS-VID-v2
- Domestic dataset mirror: https://www.modelscope.cn/datasets/lanlanlanrr/XS-VID-v2
Download
git clone https://github.com/gjhhust/YOLOFT.git
cd YOLOFT
python tools/xsvid/download_release.py --hub hf --component model --destination ./downloads
python tools/xsvid/download_release.py --hub ms --destination ./downloads
The anonymous downloader verifies checkpoint fingerprints and all 16 dataset archive hashes before preparing the legacy layout. See the repository README for the tested Python 3.10 / torch 2.4 / CUDA 11.8 environment and compiled DCNv3/correlation prerequisites; a fresh extension build is a separate step.
Contents and Protocol
| File | Purpose |
|---|---|
checkpoints/yoloft-l-xsvid-v2-paper.pt | VID; deterministic 802/802 tensor migration |
checkpoints/yoloft-l-xsvid-v2-mot.pt | Temporal backbone and MOT embedding head |
checkpoints/yoloft-l-xsvid-v2-sot.pt | Correct paper push checkpoint; TransT head |
checkpoints/transt-state-dict.pth | Portable SOT training initialization, without LTR pickle |
checkpoints/transt.pth | Original TransT artifact, retained for provenance only |
checkpoints/reid/osnet_x0_25_msmt17.pt | Track-by-Detection appearance encoder |
configs/yoloft-l-temporal.yaml | Shared temporal graph |
metrics/, manifest.json | Scores, protocol and file fingerprints |
Paper Detection uses annotations/protocols/paper_detection_test.json; new
results should use canonical annotations/test.json. Paper TAO scoring uses
sparse paper_track_id with fallback to track_id; new MOT results use canonical
track_id. Do not conflate TAO mAP with frame-level MOTA/IDF1.
Full fresh V100 inference on 2026-10-06 reproduced VID paper AP 29.3124
(canonical AP 29.3947) and all 272 SOT sequences / 107,788 frames:
AUC 59.6286, Precision@20 81.3492. The SOT SHA-256 is
c59639f868a5fdfa5474f072d657bafb072430c7386b12901e3bc1ac7a42188a.
The historical early checkpoint scoring 57.09/79.35 is not the paper checkpoint.
Fresh unified MOT inference covers all 66 test videos and reaches TAO mAP
31.5. Its subset AP_s/AP_m/AP_es/AP_rs are 21.3/32.7/16.3/40.3, versus
historical paper values 21.7/32.8/16.4/40.4. The frozen MOTChallenge protocol
gives fresh MOTA/IDF1 30.3760/54.9435, versus historical prediction replay
30.6686/55.1525. These are separate recorded results, not exact all-metric
parity. Fresh Track-by-Detection inference covers all 66 videos and reaches
TAO mAP 31.9 and MOTA/IDF1 36.9420/54.5701, versus historical paper
32.1/37.1/54.7. Rechecking against independently downloaded test annotations
preserved these scores. These numerical differences remain documented rather
than changing test thresholds to force a match.
The paper MOTChallenge scorer and frozen GT archive are in
tools/xsvid/mot_protocol_tools/ (code) and protocols/ (this package).
Canonical dataset annotations remain unchanged. Unified MOT and BoT-SORT use
different explicit historical score-filter profiles; TAO mAP is a separate
protocol. The available package is YOLOFT-L; the located YOLOFT-S candidate
has not been verified against the paper's 25.8 AP row and is not included as
a paper-reproducing S checkpoint.
Training Startup
python tools/xsvid/smoke_train.py --data-root ./downloads/XS-VID-v2 \
--model-package ./downloads/YOLOFT-XSVID-v2 --output ./startup-check \
--task all --steps 2 --frames 8 --gpu 0 --run
This is a bounded real-training startup/save test, not full paper retraining.
The portable TransT artifact contains all 430 original tensors bit-identically;
the head loads all 170/170 selected tensors without an external ltr package.
The main code README contains full training entry points and inference scoring.
Citation and Licenses
Use CITATION.cff in the code repository. Dataset: CC BY 4.0. YOLOFT runtime:
AGPL-3.0. Original TransT/OSNet and bundled dependencies retain their upstream
licenses; see repository notices. Do not treat the dataset license as a license
for all third-party code or pretrained weights.