A local-motion-based probabilistic model for visual tracking

Color-based tracking is prone to failure in situations where visually similar targets are moving in a close proximity or occlude each other. To deal with the ambiguities in the visual information, we propose an additional color-independent visual model based on the target's local motion. This model is calculated from the optical flow induced by the target in consecutive images. By modifying a color-based particle filter to account for the target's local motion, the combined color/local-motion-based tracker is constructed. We compare the combined tracker to a purely color-based tracker on a challenging dataset from hand tracking, surveillance and sports. The experiments show that the proposed local-motion model largely resolves situations when the target is occluded by, or moves in front of, a visually similar object.     

Probabilistic people tracking with appearance models and occlusion classification: The AD-HOC system

AD-HOC (Appearance Driven Human tracking with Occlusion Classification) is a complete framework for multiple people tracking in video surveillance applications in presence of large occlusions. The appearance-based approach allows the estimation of the pixel-wise shape of each tracked person even during the occlusion. This peculiarity can be very useful for higher level processes, such as action recognition or event detection. A first step predicts the position of all the objects in the new frame while a MAP framework provides a solution for best placement. A second step associates each candidate foreground pixel to an object according to mutual object position and color similarity. A novel definition of non-visible regions accounts for the parts of the objects that are not detected in the current frame, classifying them as dynamic, scene or apparent occlusions. Results on surveillance videos are reported, using in-house produced videos and the PETS2006 test set.     

A 2D/3D model-based object tracking framework

This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in (IEEE Trans. Pattern Anal. Mach. Intell. 18(2) (1996)) is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object's pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.     

A probabilistic plan recognition algorithm based on plan tree grammars

We present the PHATT algorithm for plan recognition. Unlike previous approaches to plan recognition, PHATT is based on a model of plan execution. We show that this clarifies several difficult issues in plan recognition including the execution of multiple interleaved root goals, partially ordered plans, and failing to observe actions. We present the PHATT algorithm's theoretical basis, and an implementation based on tree structures. We also investigate the algorithm's complexity, both analytically and empirically. Finally, we present PHATT's integrated constraint reasoning for parametrized actions and temporal constraints.     

Structural similarity-based object tracking in multimodality surveillance videos

This paper addresses the problem of object tracking in video sequences for surveillance applications by using a recently proposed structural similarity-based image distance measure. Multimodality surveillance videos pose specific challenges to tracking algorithms, due to, for example, low or variable light conditions and the presence of spurious or camouflaged objects. These factors often cause undesired luminance and contrast variations in videos produced by infrared sensors (due to varying thermal conditions) and visible sensors (e.g., the object entering shadowy areas). Commonly used colour and edge histogram-based trackers often fail in such conditions. In contrast, the structural similarity measure reflects the distance between two video frames by jointly comparing their luminance, contrast and spatial characteristics and is sensitive to relative rather than absolute changes in the video frame. In this work, we show that the performance of a particle filter tracker is improved significantly when the structural similarity-based distance is applied instead of the conventional Bhattacharyya histogram-based distance. Extensive evaluation of the proposed algorithm is presented together with comparisons with colour, edge and mean-shift trackers using real-world surveillance video sequences from multimodal (infrared and visible) cameras.     

Combining object and feature dynamics in probabilistic tracking

Objects can exhibit different dynamics at different spatio-temporal scales, a property that is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image features that are then used as inputs to a system for tracking the object using a global dynamic model. Approximate local dynamics may be brittle—point trackers drift due to image noise and adaptive background models adapt to foreground objects that become stationary—and constraints from the global model can make them more robust. We propose a probabilistic framework for incorporating knowledge about global dynamics into the local feature extraction processes. A global tracking algorithm can be formulated as a generative model and used to predict feature values thereby influencing the observation process of the feature extractor, which in turn produces feature values that are used in high-level inference. We combine such models utilizing a multichain graphical model framework. We show the utility of our framework for improving feature tracking as well as shape and motion estimates in a batch factorization algorithm. We also propose an approximate filtering algorithm appropriate for online applications and demonstrate its application to tasks in background subtraction, structure from motion and articulated body tracking.     

Multi-step prediction method for robust object tracking

In this paper, multi-step ahead prediction method for object tracking based on chaos theory is introduced. The chaos theory is used to preserve the information of object's movement and to model uncertainty and nonlinearity of movement in video sequences. The methodology of the algorithm includes three steps. First, adaptive pseudo-orbit data assimilation is applied to estimate the next state by using the previous states of object. Second, the ensemble members of the state are generated to predict multi-step prediction. Then, the likelihood function of members selects candidate patch for target detection using color information. The algorithm significantly reduces the prediction error because of high-order dynamical information of motion and chaotic prediction. To verify the efficiency of the tracker, the tracking algorithm is compared with the stochastic and deterministic methods under two datasets. The results demonstrate that the chaotic-based tracker outperforms other state-of-the-art methods on the abrupt motion, occlusion, and out of view. The algorithm is about two times faster than the particle filter method while the error of particle filter is about two times more than the error of the chaotic-based tracking method.     

Real-time object recognition using a modified generalized Hough transform

A technique for real-time object recognition in digital images is described. On the one hand, our approach combines robustness against occlusions, clutter, arbitrary illumination changes, and noise with invariance under rigid motion, i.e., translation and rotation. On the other hand, the computational effort is small in order to fulfill requirements of real-time applications. Our approach uses a modification of the generalized Hough transform (GHT) to improve the GHT's performance: A novel efficient limitation of the search space in combination with a hierarchical search strategy is implemented to reduce the computational effort. To meet the demands for high precision in industrial tasks, a subsequent refinement adjusts the final pose parameters. An empirical performance evaluation of the modified GHT is presented by comparing it to two standard 2D object recognition techniques.     

A general framework for the evaluation of symbol recognition methods

Performance evaluation is receiving increasing interest in graphics recognition. In this paper, we discuss some questions regarding the definition of a general framework for evaluation of symbol recognition methods. The discussion is centered on three key elements in performance evaluation: test data, evaluation metrics and protocols of evaluation. As a result of this discussion we state some general principles to be taken into account for the definition of such a framework. Finally, we describe the application of this framework to the organization of the first contest on symbol recognition in GREC’03, along with the results obtained by the participants.     

A hybrid occlusion free object tracking method using particle filter and modified galaxy based search meta-heuristic algorithm

An important problem in tracking methods is how to manage the changes in object appearance, such as illumination changes, partial/full occlusion, scale, and pose variation during the tracking process. In this paper, we propose an occlusion free object tracking method together with a simple adaptive appearance model. The proposed appearance model which is updated at the end of each time step includes three components: the first component consists of a fixed template of target object, the second component shows rapid changes in object appearance, and the third one maintains slow changes generated along the object path. The proposed tracking method not only can detect occlusion and handle it, but also it is robust against changes in the object appearance model. It is based on particle filter which is a robust technique in tracking and handles non-linear and non-Gaussian problems. We have also employed a meta-heuristic approach that is called Modified Galaxy based Search Algorithm (MGbSA), to reinforce finding the optimum state in the particle filter state space. The proposed method was applied to some benchmark videos and its results were satisfactory and better than results of related works.     

50 Years of object recognition: Directions forward

Object recognition systems constitute a deeply entrenched and omnipresent component of modern intelligent systems. Research on object recognition algorithms has led to advances in factory and office automation through the creation of optical character recognition systems, assembly-line industrial inspection systems, as well as chip defect identification systems. It has also led to significant advances in medical imaging, defence and biometrics. In this paper we discuss the evolution of computer-based object recognition systems over the last fifty years, and overview the successes and failures of proposed solutions to the problem. We survey the breadth of approaches adopted over the years in attempting to solve the problem, and highlight the important role that active and attentive approaches must play in any solution that bridges the semantic gap in the proposed object representations, while simultaneously leading to efficient learning and inference algorithms. From the earliest systems which dealt with the character recognition problem, to modern visually-guided agents that can purposively search entire rooms for objects, we argue that a common thread of all such systems is their fragility and their inability to generalize as well as the human visual system can. At the same time, however, we demonstrate that the performance of such systems in strictly controlled environments often vastly outperforms the capabilities of the human visual system. We conclude our survey by arguing that the next step in the evolution of object recognition algorithms will require radical and bold steps forward in terms of the object representations, as well as the learning and inference algorithms used.     

An active contour model for object tracking using the previous contour

An active contour model is proposed for object tracking using prior information. Conventional algorithms have many problems when applied in object tracking. The proposed active contour algorithm, a model using an edge of an adapted color feature, not only modifies the internal energy function of the conventional algorithm to extend the search range and reduce the computational burden, but also modifies the external energy function to reduce the edge candidates of the object. The algorithm searches normally and uses dynamic programming to solve the energy minimization problem. The main drawbacks of a conventional snake algorithm, i.e., shrinking, a limited search range, sensitivity to outliers, are improved with the proposed algorithm. We illustrate the effectiveness of our scheme using some tracking examples. This work was presented, in part, at the Seventh International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

Visual object recognition using probabilistic kernel subspace similarity

Probabilistic subspace similarity-based face matching is an efficient face recognition algorithm proposed by Moghaddam et al. It makes one basic assumption: the intra-class face image set spans a linear space. However, there are yet no rational geometric interpretations of the similarity under that assumption. This paper investigates two subjects. First, we present one interpretation of the intra-class linear subspace assumption from the perspective of manifold analysis, and thus discover the geometric nature of the similarity. Second, we also note that the linear subspace assumption does not hold in some cases, and generalize it to nonlinear cases by introducing kernel tricks. The proposed model is named probabilistic kernel subspace similarity (PKSS). Experiments on synthetic data and real visual object recognition tasks show that PKSS can achieve promising performance, and outperform many other current popular object recognition algorithms.     

A general shape context framework for object identification

A general shape context framework is proposed for object/image retrieval in occluded and cluttered environment with hundreds of models as the potential matches of an input. The approach is general since it does not require separation of input objects from complex background. It works by first extracting consistent and structurally unique local neighborhood information from inputs or models, and then voting on the optimal matches. Its performance degrades gracefully with respect to the amount of structural information that is being occluded or lost. The local neighborhood information applicable to the system can be shape, color, texture feature, etc. Currently, we employ shape information only. The mechanism of voting is based on a novel hyper cube based indexing structure, and driven by dynamic programming. The proposed concepts have been tested on database with thousands of images. Very encouraging results have been obtained.     

A framework for heading-guided recognition of human activity

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. Heading-guided recognition (HGR) is proposed as an efficient method for adaptive classification of activity. The HGR approach is demonstrated using “motion history images” that are then recognized via a mixture-of-Gaussians classifier. The system is tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. In addition, experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.     

Fast object recognition using dynamic programming from combination of salient line groups

This paper presents a new method of grouping and matching line segments to recognize objects. We propose a dynamic programming-based formulation extracting salient line patterns by defining a robust and stable geometric representation that is based on perceptual organizations. As the endpoint proximity, we detect several junctions from image lines. We then search for junction groups by using the collinear constraint between the junctions. Junction groups similar to the model are searched in the scene, based on a local comparison. A DP-based search algorithm reduces the time complexity for the search of the model lines in the scene. The system is able to find reasonable line groups in a short time.     

A graph-based, multi-resolution algorithm for tracking objects in presence of occlusions

One of the main difficult problem in video analysis is to track moving objects during a video sequence, especially in presence of occlusions. Unfortunately, almost all the different approaches work on a pixel-by-pixel basis, yielding them unusable in real-time situations as well as not much expressive at a semantic level of the video. We present in this paper a novel method of tracking objects through occlusions that exploits the wealth of information due to the spatial coherence between pixels, using a graph-based, multi-resolution representation of the moving regions. The experimental results show that the approach is promising.     

Automatic model-based 3D object recognition by combining feature matching with tracking

We propose a vision-based robust automatic 3D object recognition, which provides object identification and 3D pose information by combining feature matching with tracking. For object identification, we propose a robust visual feature and a probabilistic voting scheme. An initial object pose is estimated using correlations between the model image and the 3D CAD model, which are predefined, and the homography, byproduct of the identification. In tracking, a Lie group formalism is used for robust and fast motion computation. Experimental results show that object recognition by the proposed method improves the recognition range considerably. Sungho Kim received the B.S. degree in Electrical Engineering from Korea University, Korea in 2000 and the M.S. degree in Electrical Engineering and Computer Science from Korea Advanced Institute of Science and Technology, Korea in 2002. He is currently pursuing his Ph.D. at the latter institution, concentrating on 3D object recognition and tracking. In So Kweon received the Ph.D. degree in robotics from Carnegie Mellon University, Pittsburgh, PA, in 1990. Since 1992, he has been a Professor of Electrical Engineering at KAIST. His current research interests include human visual perception, object recognition, real-time tracking, vision-based mobile robot localization, volumetric 3D reconstruction, and camera calibration. He is a member of the IEEE, and Korea Robotics Society (KRS).