Calisthenics pose completion v4.1
Browser inference bundle for evidence-gated planche display geometry in CaliCombos AI Annotate. This is an active, bounded display refinement, not a universal replacement for every visible or occluded joint. Raw observations and movement annotations remain separate. No server inference, clip upload, clip identifier, filename, timestamp, or stored reference animation is used at inference.
Models
| File | Function | Download bytes |
|---|---|---|
| person-detector.onnx | HumanArt YOLOX Tiny target-person crop | 10,181,266 |
| pose-evidence.onnx | Adapted RTMW whole-body image predictor, per-joint source-region support, leg configuration | 65,232,135 |
| body-completion.onnx | Camera-normalized sparse-observation MHR body geometry; 23 connected body/head/foot points | 472,826 |
Image models use float16 weight storage and float32 activations. The small body model uses float32. Read manifest.json for immutable asset digests, tensor conventions, and scope. The application supplies ONNX Runtime Web WASM, preprocessing, identity selection, hand observations, camera normalization, confidence checks, source-ray anchoring, temporal display filtering, and the separate humanoid mesh decoder. Loading the body ONNX file alone is not the complete pipeline.
The body input is [1,12,9]: image rays (masked if unsupported), prior relative world XYZ, source-support mask, depth confidence, normalized camera root distance, extended-leg probability. Body input order is left/right shoulder, elbow, wrist, hip, knee, ankle. Output is [1,23,3], MHR landmarks 0–20 plus left and right wrists. Log variance is a training uncertainty output, not a calibrated visibility probability.
Evidence and limits
- The previously false cropped-ankle supports are 0.479/0.464, below the application's 0.65 support threshold. Crop-edge joints are separately rejected. Category visibility cannot override joint support.
- Source-region image coordinates retain 40/67 validation and 46/65 test agreements on identity-resolvable review frames. These denominators differ from older full-image-crop reports and must not be compared as a claimed accuracy increase.
- Whole-clip browser reconstruction: 130 frames, 100 accepted planche completions, median knees 172.14°/173.42°, hold retained at 12.5 seconds. This clip's corrected offline geometry is a training source; the result is reconstruction, not independent real-world 3D validation.
- All raw observations were byte-for-byte equal to the baseline. Existing upper-body image anchors are preserved; supported leg corrections belong to the separate display fit.
- Crowded regression: 178 frames, no accepted replacement and unchanged annotations. This verifies abstention, not successful crowded-scene recovery.
- Native versus browser image inference has identical body peaks. Editor tests use desktop Chrome, including a 390×844 viewport. Physical phones have not been tested.
Identity, planche context, extension, visible-chain, bounds and geometry checks can withhold completion. Unsupported movements retain the prior pipeline. Hidden joints are inferred and labeled as estimates; they are not measured or assigned observed visibility. This does not establish calibrated metric 3D, general anatomical accuracy, fingertip load or supination correctness. The accompanying validation.json records the scoped checks.
Training
The image evidence heads use positive and negative candidate-to-source-region agreement, with unknown categories masked. The coordinate predictor is frozen during evidence training. The body model uses MHR-generated poses across camera orientations, grouped occlusions, preservation examples, and corrected whole-video fit geometry. A final distillation pass uses actual camera-normalized browser inputs and the corrected fit to learn leg spacing and foot orientation. Same-clip temporal reconstruction subsets select optimization length; they are not independent validation. Broader synthetic pose replay regularizes that pass.
Licenses and provenance
- Image pose backbone: OpenMMLab MMPose / RTMW (
rtmw-dw-l-m_simcc-cocktail14_270e-256x192_20231122.onnx); see LICENSE-MMPOSE.txt. Modified by source-region training, evidence/configuration heads and float16 weight packing. - Detector: YOLOX Tiny HumanArt (
yolox_tiny_8xb8-300e_humanart-6f3252f9.onnx), distributed through RTMLib; see LICENSE-YOLOX.txt. Modified by float16 weight packing. - Geometry: Meta MHR and corrected SAM 3D Body fit outputs. See LICENSE-MHR.txt and LICENSE-SAM.txt. The compact student is newly trained; the original multi-gigabyte SAM checkpoint is not distributed here. Use of SAM materials is acknowledged.
Component-specific upstream terms are preserved; this bundle does not relicense SAM materials under Apache-2.0. No private source videos, review images, per-frame annotations, caches or credentials are included.