RaCo-ALIKED — extractor half (ONNX + TensorRT engines)
Keypoint detection (RaCo) + 128-D descriptors (ALIKED), as a standalone ONNX
graph, for vision-rt's vrt-raco-aliked crate.
images (B,3,H,W) f32 H,W multiples of 32, RGB in [0,1]
-> keypoints (B,K,2) f32 model-resolution pixels (x,y)
-> normalized_keypoints (B,K,2) f32 long-edge normalised, matcher input
-> descriptors (B,K,128) f32 already L2-normalised
The matcher half lives in kornia/lightglue-onnx.
Why these files exist
Upstream (fabio-sim/LightGlue-ONNX)
publishes only a fused extractor+matcher graph. That graph emits int64, has
data-dependent output shapes, and never exposes descriptors — so it cannot feed a
descriptor bank or a relocalization database, and a fused forward can only ever match the
two images handed to it.
All three problems share one root: LightGlue computes per-query matches and only then
compacts them with a single NonZero. That NonZero is the only TensorRT-hostile op in
the whole graph. Cutting upstream of it yields fully static shapes; cutting again at the
extractor/matcher seam yields two independently reusable halves. These files are that cut,
produced by
split_raco_pipeline.py.
The split is verified against the fused model under onnxruntime — keypoints, match set and scores all identical.
K is baked in, and changes the graph
K is fixed at export. It is not just a keypoint budget: at K ≥ 3072 RaCo's learned
ranker is omitted entirely, which roughly halves extraction cost while returning 3× the
keypoints. Measured on a Jetson Orin Nano (MAXN_SUPER, TRT 10.3.0.30, fp16, 640²):
| K | ranker | extract / image |
|---|---|---|
| 512 | dense | 49.1 ms |
| 1024 | boundary | 55.2 ms |
| 3072 | bypass | 28.5 ms |
Extraction is non-monotonic in K. The trade is accuracy under rotation: inliers against a ground-truth affine drop from ~98% (k1024) to ~93% (k3072) at 90°, but k3072 returns ~2.6× more matches, so it yields more correct correspondences in absolute terms. Note the matcher scales the other way (O(K²)) — see the matcher repo.
TensorRT engines
Engines are machine-locked: they only load on the exact TensorRT version and GPU
architecture they were built for (here trt10.3.0.30, sm87 — JetPack 6 on Orin). On any
other machine, build from the ONNX instead; vrt-hub does this automatically.
Each engine here is built at the shape profile RaCoAliked::engine_profile() declares
(min 1x3x256x256, opt 2x3x512x512, max 2x3x640x640). That matters: vrt-hub selects a
prebuilt on TensorRT version, SM and precision alone and does not check the shape
profile, so an engine built at a different profile would be served and then reject frames
it cannot handle.
The released graph targets TRT 10.16 / CUDA 13, whose ONNX parser constant-folds
aggressively. TRT 10.3 does not, and rejects two torch-dynamo constructs (Conv with a
computed zero bias, Reduce* with computed axes). The split script bakes both into
initializers — without which the unmodified fused model also fails to parse on TRT 10.3.
Licences and model credit
These artifacts are derived from three separately licensed upstreams and carry all
three obligations. See LICENSE-NOTICE.md.
| Component | Source | Licence |
|---|---|---|
| RaCo detector + ranker | cvg/RaCo | Apache-2.0 |
| ALIKED descriptors | Shiaoming/ALIKED | BSD-3-Clause |
| ONNX export tooling | fabio-sim/LightGlue-ONNX | Apache-2.0 |
All model credit belongs to the original authors:
- RaCo — Shenoi, Lindenberger, Sarlin, Pollefeys, "RaCo: Ranking and Covariance for Practical Learned Keypoints", 3DV 2026, arXiv:2602.15755.
- ALIKED — Zhao et al., "ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation", IEEE TIM 2023.
The TensorRT graph optimizations baked into these files (hierarchical chunked TopK, BatchNorm folding, native SELU + logit-space NMS, ranker boundary-reranking, the DeformConv→GridSample rewrite, and the integer-floor-div fp16 fix) are fabio-sim's work and are preserved verbatim by the split.