deeperfly detectors

Pretrained 2D keypoint detectors for deeperfly, markerless 3D pose estimation of Drosophila from multi-camera rigs. They include the detectors the example run configs in the deeperfly repository name, all trained in the Ramdya lab at EPFL on hand labels from two rigs.

A real-time octacam detector is in training and will be added here.

Expected input

Each detector expects frames like these (the first two are the example clips in the deeperfly repository); the dashed boxes are its windows, and the skeleton is what it predicts. Pretrained models in the deeperfly docs describes both rigs.

octacam

hexaview, one fly

hexaview, two flies

Each folder holds the checkpoint (.pth), the skeleton it predicts (fly38.toml), the training recipe (config.toml), the training report (report.json), SHA256SUMS, and a README.md written when the model was exported. The paths in those READMEs are the lab's; use the instructions below instead.

Download

From the root of a deeperfly clone, where the example configs expect them:

uv run hf download tkclam/deeperfly-models --local-dir models

One model only, such as the octacam detector:

uv run hf download tkclam/deeperfly-models --local-dir models \
  --include "261008_mvt_dinov3b_384_octacam_fly38/*"

Check a download with sha256sum -c SHA256SUMS inside its folder.

Use

Name the checkpoint in a run config's [pose2d] table:

[pose2d]
weights = ["models/261008_mvt_dinov3b_384_octacam_fly38/mvt_dinov3b_384_octacam_fly38.pth"]

Paths resolve against the config file. See the deeperfly quickstart for a full run on the bundled example clips.

Scope and limitations

The detectors were trained on recordings from two rigs in one lab, with every label used for training (no held-out set). The octacam detector's twin, trained with four recordings held out, scores 3.25 px median and 4.49 px mean on their 9,269 labeled cells (93.7% within 10 px). On the bundled example clips, the octacam detector reprojects at 1.46 px median after bundle adjustment and the hexaview detector at 3.00 px median through the recording's calibration; that is agreement across views, not error against labels. The two-fly detector's twin from the round before, trained with recordings held out, put 4.0% of keypoints more than 15 px from the label in the direct views and 9.2% in the prism views on 35 held-out two-fly frames (5.8% and 14.7% for the single-fly detector's twin). They are not expected to transfer to other rigs, magnifications, or lighting without retraining. deeperfly train fits a detector on your own labels, and these checkpoints can serve as a starting point.

License

• The DINOv3-based detectors are derivatives of Meta's DINOv3 weights and are distributed under the DINOv3 License, including its use restrictions.
• The EfficientViT-based locators are distributed under the Apache License 2.0, the license of its ImageNet-pretrained backbone.

Citation

Please cite deeperfly (see its README) and DINOv3:

@misc{simeoni2025dinov3,
  title         = {{DINOv3}},
  author        = {Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and others},
  year          = {2025},
  eprint        = {2508.10104},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
}