GeoPatch: Learning Geodesic-Aware Features from RGB-D Images
Computer Vision and Image Understanding (CVIU 2022)
Extracting geodesic features. Due to the invariance property of the geodesic distance, the pixel intensities lying on the same isocurves do not change after the deformations and can be sampled to compute an invariant descriptor. Descriptors A and B are still similar (pixels indicated by squares in red, blue, and green lie on the same isocurve after the deformation).
Abstract
Most of the existing handcrafted and learning-based local descriptors are still at best approximately invariant to affine image transformations, often disregarding deformable surfaces. In this paper, we take one step further by proposing a new approach to compute descriptors from RGB-D images (where RGB refers to the pixel color brightness and D stands for depth information) that are invariant to isometric non-rigid deformations, as well as to scale changes and rotation. Our proposed description strategies are grounded on the key idea of learning feature representations on undistorted local image patches using surface geodesics. We design two complementary local descriptors strategies to compute geodesic-aware features efficiently: one efficient binary descriptor based on handcrafted binary tests (named GeoBit), and one learning-based descriptor (GeoPatch) with convolutional neural networks (CNNs) to compute features. In different experiments using real and publicly available RGB-D data benchmarks, they consistently outperforms state-of-the-art handcrafted and learning-based image and RGB-D descriptors in matching scores, as well as in object retrieval and non-rigid surface tracking experiments, with comparable processing times. We also provide to the community a new dataset with accurate matching annotations of RGB-D images of different objects (shirts, cloths, paintings, bags), subjected to strong non-rigid deformations, for evaluation benchmark of deformable surface correspondence algorithms. Code and weights are available below!
Links |
Team
Guilherme Augusto Potje
PhD Candidate
Renato José Martins
Professor at Université de Bourgogne
Felipe Cadar Chamone
PhD Student