Physical Scale Keypoints: Matching and Registration for Combined Intensity/Range Images

We present a new framework for detecting, describing, and matching keypoints in combined range-intensity data, resulting in what we call physical scale keypoints. We first produce an image mesh by backprojecting associated 2D intensity images onto the 3D range data. We detect and describe keypoints on the image mesh using an analogue of the SIFT algorithm for images with two key modifications: the process is made insensitive to viewpoint and structural discontinuities using a novel bilinear filter, and a physical scale space is constructed that exploits the reliable range measurements. Keypoints are matched between scans only when their physical scales agree, avoiding many potential false matches. Finally, the matches are rank-ordered using a new quality measure and supplied to a registration algorithm that refines each match into a rigid transformation for the entire scan pair. We report experimental results on keypoint detection and matching and range scan registration and verification in a set of difficult real-world scan pairs, showing that the new physical scale keypoints are demonstrably better than a competing approach based on backprojected SIFT keypoints.