Feature Detection Performance Based Benchmarking of Motion Deblurring Methods: Applications to Vision for Legged Robots

Dexterous legged robots can move on variable terrain at high speeds. The locomotion of these legged platforms on such terrain causes severe oscillations of the robot body depending on the surface and locomotion speed. Camera sensors mounted on such platforms experience the same disturbances, hence resulting in motion blur. This is a particular corruption of the image and results in information loss further resulting in degradation or loss of important image features. Although motion blur is a significant problem for legged mobile robots, it is of more general interest since it is present in many other handheld/mobile camera applications. Deblurring methods exist in the literature to compensate for blur, however most proposed performance metrics focus on the visual quality of compensated images. From the perspective of computer vision algorithms, feature detection performance is an essential factor that determines vision performance. In this study, we claim that existing image quality based metrics are not suitable to assess the performance of deblurring algorithms when the output is used for computer vision in general and legged robotics in particular. For comparatively evaluating deblurring algorithms, we define a novel performance metric based on the feature detection accuracy on sharp and deblurred images. We rank these algorithms according to the new metric as well as image quality based metrics from the literature and experimentally demonstrate that existing metrics may not be good indicators of algorithm performance, hence good selection criteria for computer vision application. Additionally, noting that a suitable data set to evaluate the effects of motion blur and its compensation for legged platforms is lacking in the literature, we develop a comprehensive multi-sensor data set for that purpose. The data set consists of monocular image sequences collected in synchronization with a low cost MEMS gyroscope, an accurate fiber optic gyroscope and an externally measured ground truth motion data. We make use of this data set for an extensive benchmarking of prominent motion deblurring methods from the literature in terms of existing and the proposed feature based metric.