A Constraint-Satisfaction Approach for 3-D Object Recognition by Integrating 2-D and 3-D Data

We present a framework for recognition of 3D objects by integrating 2D and 3D sensory data. The major thrust of this work is to efficiently utilize all relevant data, both 2D and 3D, in the early stages of recognition, in order to reduce the computational requirements of the recognition process. To achieve this goal, we formulate the problem as a constraint–satisfaction problem (CSP). Rather than directly solving the CSP, a problem of exponential complexity, we only enforce local consistency in low-order polynomial time. This step of local-consistency enforcement can significantly decrease the computational load on subsequent recognition modules by (1) significantly reducing the uncertainty in the correspondence between scene and model features and (2) eliminating many erroneous model objects and irrelevant scene features. A novel method is presented for efficiently constructing a CSP corresponding to a combination of 2D and 3D scene features. Performance of the proposed framework is demonstrated using simulated and real experiments involving visual (2D) and tactile (3D) data.