GEM-X
GEM-X is NVIDIA's monocular whole-body motion estimator built around the SOMA-X parametric body model.
- Paper lineage: GEM: A Generalist Model for Human Motion
- Official source: NVlabs/GEM-X
- Motius checkpoint: ZeyuLing/Motius-GEM-X
- Pinned source revision:
32992550dba114c62243fb55e361311972dce8f9 - Pinned SOMA-X revision:
e0f8ff0ecfa3edbbb6058b1e0f08822ee2f84ee5 - Checkpoint SHA-256:
4c1f85ca8c1e11e6588aead49fbc024bf660708def670043e0b537c101ee298e
Tasks: Monocular Motion Capture
Supported Tasks
| Task | Public API | Input | Output |
|---|---|---|---|
| Monocular Motion Capture | infer_monocular_motion_capture | RGB video | MonocularCaptureResult |
Checkpoint
The Motius Hugging Face artifact is complete: GEM-X, SAM-3D-Body, DINOv3,
ViTPose, YOLOX, MHR, SOMA-X identity/corrective assets, and normalization
statistics are stored under their expected paths with SHA-256 provenance.
Pipeline.from_pretrained therefore needs no source checkout or second model
download.
Motion Representation
GEM-X natively predicts SOMA-77:
- 77-joint axis-angle pose;
- 45 identity coefficients;
- 69 global/body-part scale parameters;
- camera and world root translations;
- 77 named joints and 4,505-vertex low-LOD SOMA meshes.
Motius keeps SOMA-X native. It does not manufacture SMPL vertices from a
different topology. Cross-model 3DPW evaluation uses the audited
common_hmr15_named_v1 joint subset; PVE is reported as unavailable.
Usage
Create an isolated environment without cloning GEM-X:
python3.10 -m venv outputs/envs/gem-x
outputs/envs/gem-x/bin/pip install -e ".[gem-x]"
Run the standard task API:
from pathlib import Path
from motius.motion.representation.monocular_capture import (
save_monocular_capture_result,
)
from motius.pipelines.gem_x import GemXPipeline
pipeline = GemXPipeline.from_pretrained(
"ZeyuLing/Motius-GEM-X",
bundle_kwargs={
"python_executable": "outputs/envs/gem-x/bin/python",
},
)
result = pipeline.infer_monocular_motion_capture(
"input.mp4",
output_root="outputs/gem_x/run_001",
materialize_geometry=True,
render=True,
)
save_monocular_capture_result(
result,
Path("outputs/gem_x/run_001/result.npz"),
)
render=True requests the upstream keypoint, in-camera, and world previews.
Leave it off for evaluation and batch inference.
Demo
This 768px, 30 FPS preview renders the world-space SOMA-X mesh returned by the public Motius pipeline.
Evaluation Results
Protocol: 3dpw_test_camera_v1, all 24 test videos and 37 official person
tracks, evaluated on common_hmr15_named_v1.
| Coverage | MPJPE ↓ | PA-MPJPE ↓ | Acceleration ↓ |
|---|---|---|---|
| 100.00% | 84.38 mm | 53.20 mm | 5.616 m/s² |
The official demo emits an identity camera trajectory when no external visual odometry is supplied. These are camera-space metrics; world-space ranking is unavailable for that run.
Stage Parity
The strict gate compares all persisted boundaries. Deterministic fields are
bitwise exact. The official contact IK/SOMA CUDA postprocess is non-deterministic
across independent processes, so only its 15 explicitly named descendants use
a hard 3e-6 absolute-error ceiling. No global tolerance is applied.
| Boundary | Fields | Requirement | Result |
|---|---|---|---|
| Tracking | 2 | exact | pass |
| Keypoints and camera | 2 | exact | pass |
| Visual features | 2 | exact | pass |
| Complete model input | 18 | exact | pass |
| Raw network output and deterministic decoded fields | 27 | exact | pass |
| Contact-postprocessed model fields | 7 | atol ≤ 3e-6 | max 6.71e-7 |
| SOMA geometry | 4 | atol ≤ 3e-6 | max 2.38e-6 |
| Public result | 9 | exact except 4 descendants | max 9.54e-7 |
| Total | 71 | field-scoped policy | pass |
python tools/verify_monocular_pipeline_parity.py \
--profile gem-x \
--reference outputs/parity/gem_x/reference_trace.npz \
--candidate outputs/parity/gem_x/motius_trace.npz
License
GEM-X source is Apache-2.0 and the public weights use the NVIDIA Open Model License. SAM-3D-Body, DINOv3, SOMA-X, MHR, and their assets retain their own terms. See the GEM-X attributions and the packaged third-party notices before use.
Direct Loading
from motius import Pipeline
pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-GEM-X")