3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients

This contribution addresses the problem of pose estimation and tracking of vehicles in image sequences from traffic scenes recorded by a stationary camera. In a new algorithm, the vehicle pose is estimated by directly matching polyhedral vehicle models to image gradients without an edge segment extraction process. The new approach is significantly more robust than approaches that rely on feature extraction since the new approach exploits more information from the image data. We successfully tracked vehicles that were partially occluded by textured objects, e.g., foliage, where a previous approach based on edge segment extraction failed. Moreover, the new pose estimation approach is also used to determine the orientation and position of the road relative to the camera by matching an intersection model directly to image gradients. Results from various experiments with real world traffic scenes are presented.     

A Framework for Model-Based Tracking Experiments in Image Sequences

Motris, an integrated system for model-based tracking research, has been designed modularly to study the effects of algorithmic variations on tracking results. Motris attempts to avoid introducing bias into the relative assessment of alternative approaches. Such a bias may be caused by differences of implementation and parameterization if the component approaches are evaluated in separate testing environments. Tracking results are evaluated automatically on a significant test sample in order to quantify the effects of different combinations of alternatives. The Motris system environment thus allows an in-depth comparison between the so-called ‘Edge-Element Association’ approach documented in Haag and Nagel (1999) and the more recent ‘Expectation-Maximization’ approach reported by Pece and Worrall (2002).     

Combination of Edge Element and Optical Flow Estimates for 3D-Model-Based Vehicle Tracking in Traffic Image Sequences

A model-based vehicle tracking system for the evaluation of inner-city traffic video sequences has been systematically tested on about 15 minutes of real world video data. Methodological improvements during preparatory test phases affected—among other changes—the combination of edge element and optical flow estimates in the measurement process and a more consequent exploitation of background knowledge. The explication of this knowledge in the form of models facilitates the evaluation of video data for different scenes by exchanging the scene-dependent models. An extensive series of experiments with a large test sample demonstrates that the current version of our system appears to have reached a relative optimum: further interactive tuning of tracking parameters does no longer promise to improve the overall system performance significantly. Even the incorporation of further knowledge regarding vehicle and scene geometry or illumination has to cope with an increasing level of interaction between different knowledge sources and system parameters. Our results indicate that model-based tracking of rigid objects in monocular image sequences may have to be reappraised more thoroughly than anticipated during the recent past.     

On change detection and displacement vector estimation in image sequences

Recent developments in the areas of displacement vector estimation as well as dissimilarity grading by a maximum likelihood ratio can be related to each other quantitatively in such a way that dissimilarity grading is reduced to interframe displacement estimation.     

Model-Based Stereo-Tracking of Non-Polyhedral Objects for Automatic Disassembly Experiments

Automatic disassembly tasks in the engine compartment of a used car constitute a challenge for control of a disassembly robot by machine vision. Experience in exploratory experiments under such conditions forced us to abandon data-driven aggregation of edge elements into straight-line data segments in favor of a direct association of individual edge elements with model segments obtained from scene domain models of tools and workpieces. In addition, we had to switch from a conventional single camera hand-eye configuration to a movable stereo-configuration mounted on a separate observer robot. A generalisation of our model-based tracking includes the parameters, which characterize the relative pose of one camera with respect to the other one of the stereo-camera set-up, into the set of parameters to be re-estimated for each new stereo image pair. This results in a continuous re-calibration during a relative movement between stereo-camera set-up and tracked objects. Our approach had to be extended further in order to cope with non-polyhedral objects.The methodological improvements of machine vision in the course of this research are treated in detail. We discuss, moreover, the systematic trading-off of computational resources for increased robustness which is vital for visual control of automatic disassembly robots.     

Cognitive visual tracking and camera control

Cognitive visual tracking is the process of observing and understanding the behavior of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.     

Optical flow estimation and the interaction between measurement errors at adjacent pixel positions

In order to estimate both components of optical flow as well as their first spatio-temporal derivatives, it is postulated that the Optical Flow Constraint Equation (OFCE) is valid in a spatio-temporal neighborhood of pixels. So far, it has been tacitly assumed that the partial derivatives of the gray value distribution—which are required for this approach at the pixel positions involved—are independent from each other. It is shown how dropping this assumption affects the estimation procedure, based on well established approaches of estimation theory. The insight gained thereby is used to develop an approach towards merging image regions based on the compatibility of optical flow estimates obtained within the regions considered for merger.This contribution is dedicated to Dr. h.c. Lothar Späth who throughout many years furthered the Fakultät für Informatik der Universität Karlsruhe (TH) and substantially helped to provide a much-needed new building in which to teach and to do research has been and continues to be a source of pleasure.