A system for learning statistical motion patterns

Analysis of motion patterns is an effective approach for anomaly detection and behavior prediction. Current approaches for the analysis of motion patterns depend on known scenes, where objects move in predefined ways. It is highly desirable to automatically construct object motion patterns which reflect the knowledge of the scene. In this paper, we present a system for automatically learning motion patterns for anomaly detection and behavior prediction based on a proposed algorithm for robustly tracking multiple objects. In the tracking algorithm, foreground pixels are clustered using a fast accurate fuzzy k-means algorithm. Growing and prediction of the cluster centroids of foreground pixels ensure that each cluster centroid is associated with a moving object in the scene. In the algorithm for learning motion patterns, trajectories are clustered hierarchically using spatial and temporal information and then each motion pattern is represented with a chain of Gaussian distributions. Based on the learned statistical motion patterns, statistical methods are used to detect anomalies and predict behaviors. Our system is tested using image sequences acquired, respectively, from a crowded real traffic scene and a model traffic scene. Experimental results show the robustness of the tracking algorithm, the efficiency of the algorithm for learning motion patterns, and the encouraging performance of algorithms for anomaly detection and behavior prediction.     

Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space

This paper proposes a novel technique for clustering and classification of object trajectory-based video motion clips using spatiotemporal function approximations. Assuming the clusters of trajectory points are distributed normally in the coefficient feature space, we propose a Mahalanobis classifier for the detection of anomalous trajectories. Motion trajectories are considered as time series and modelled using orthogonal basis function representations. We have compared three different function approximations – least squares polynomials, Chebyshev polynomials and Fourier series obtained by Discrete Fourier Transform (DFT). Trajectory clustering is then carried out in the chosen coefficient feature space to discover patterns of similar object motions. The coefficients of the basis functions are used as input feature vectors to a Self- Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Our proposed techniques are validated on three different datasets – Australian sign language, hand-labelled object trajectories from video surveillance footage and real-time tracking data obtained in the laboratory. Applications to event detection and motion data mining for multimedia video surveillance systems are envisaged.     

Real-Time Physics-Based 3D Biped Character Animation Using an Inverted Pendulum Model

We present a physics-based approach to generate 3D biped character animation that can react to dynamical environments in real time. Our approach utilizes an inverted pendulum model to online adjust the desired motion trajectory from the input motion capture data. This online adjustment produces a physically plausible motion trajectory adapted to dynamic environments, which is then used as the desired motion for the motion controllers to track in dynamics simulation. Rather than using Proportional-Derivative controllers whose parameters usually cannot be easily set, our motion tracking adopts a velocity-driven method which computes joint torques based on the desired joint angular velocities. Physically correct full-body motion of the 3D character is computed in dynamics simulation using the computed torques and dynamical model of the character. Our experiments demonstrate that tracking motion capture data with real-time response animation can be achieved easily. In addition, physically plausible motion style editing, automatic motion transition, and motion adaptation to different limb sizes can also be generated without difficulty.     

Natural metrics and least-committed priors for articulated tracking

In articulated tracking, one is concerned with estimating the pose of a person in every frame of a film. This pose is most often represented as a kinematic skeleton where the joint angles are the degrees of freedom. Least-committed predictive models are then phrased as a Brownian motion in joint angle space. However, the metric of the joint angle space is rather unintuitive as it ignores both bone lengths and how bones are connected. As Brownian motion is strongly linked with the underlying metric, this has severe impact on the predictive models. We introduce the spatial kinematic manifold of joint positions, which is embedded in a high dimensional Euclidean space. This Riemannian manifold inherits the metric from the embedding space, such that distances are measured as the combined physical length that joints travel during movements. We then develop a least-committed Brownian motion model on the manifold that respects the natural metric. This model is expressed in terms of a stochastic differential equation, which we solve using a novel numerical scheme. Empirically, we validate the new model in a particle filter based articulated tracking system. Here, we not only outperform the standard Brownian motion in joint angle space, we are also able to specialise the model in ways that otherwise are both difficult and expensive in joint angle space.     

VICs: A modular HCI framework using spatiotemporal dynamics

Many vision-based human-computer interaction systems are based on the tracking of user actions. Examples include gaze tracking, head tracking, finger tracking, etc. In this paper, we present a framework that employs no user tracking; instead, all interface components continuously observe and react to changes within a local neighborhood. More specifically, components expect a predefined sequence of visual events called visual interface cues (VICs). VICs include color, texture, motion, and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-based HCI paradigm and then focus specifically on an appearance-based system in which a hidden Markov model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button push with a 96% success rate.Published online: 19 November 2004     

Automatic acquisition and initialization of articulated models

Tracking, classification and visual analysis of articulated motion is challenging because of the difficulties involved in separating noise and variabilities caused by appearance, size and viewpoint fluctuations from task-relevant variations. By incorporating powerful domain knowledge, model-based approaches are able to overcome these problem to a great extent and are actively explored by many researchers. However, model acquisition, initialization and adaptation are still relatively under-investigated problems, especially for the case of single-camera systems. In this paper, we address the problem of automatic acquisition and initialization of articulated models from monocular video without any prior knowledge of shape and kinematic structure. The framework is applied in a human-computer interaction context where articulated shape models have to be acquired from unknown users for subsequent limb tracking. Bayesian motion segmentation is used to extract and initialize articulated models from visual data. Image sequences are decomposed into rigid components that can undergo parametric motion. The relative motion of these components is used to obtain joint information. The resulting components are assembled into an articulated kinematic model which is then used for visual tracking, eliminating the need for manual initialization or adaptation. The efficacy of the method is demonstrated on synthetic as well as natural image sequences. The accuracy of the joint estimation stage is verified on ground truth data.Correspondence to: N. Krahnstoever     

Representing cyclic human motion using functional analysis

We present a robust automatic method for modeling cyclic 3D human motion such as walking using motion-capture data. The pose of the body is represented by a time-series of joint angles which are automatically segmented into a sequence of motion cycles. The mean and the principal components of these cycles are computed using a new algorithm that enforces smooth transitions between the cycles by operating in the Fourier domain. Key to this method is its ability to automatically deal with noise and missing data. A learned walking model is then exploited for Bayesian tracking of 3D human motion.     

Motion imitation with a handheld camera

In this paper, we present a novel method to extract motion of a dynamic object from a video that is captured by a handheld camera, and apply it to a 3D character. Unlike the motion capture techniques, neither special sensors/trackers nor a controllable environment is required. Our system significantly automates motion imitation which is traditionally conducted by professional animators via manual keyframing. Given the input video sequence, we track the dynamic reference object to obtain trajectories of both 2D and 3D tracking points. With them as constraints, we then transfer the motion to the target 3D character by solving an optimization problem to maintain the motion gradients. We also provide a user-friendly editing environment for users to fine tune the motion details. As casual videos can be used, our system, therefore, greatly increases the supply source of motion data. Examples of imitating various types of animal motion are shown.     

Joint tracking and classification based on aerodynamic model and radar cross section

We present interacting multiple model regularized particle filter for the X-band active surveillance radar to jointly track and classify air threaten targets. The actual aerodynamic equations for flight are used as motion model, and automatic target classification is made possible by the inclusion of radar cross section in the measurement vector. Thus, tracking and classification are closely coupled, giving full play to the advantages of joint tracking and classification. The proposed methodologies show good performance according to simulations.     

On the robust control of two manipulators holding a rigid object

In this paper, a control architecture is developed for the closed chain motion of two six-joint manipulators holding a rigid object in a three-dimensional workspace. Dynamic and kinematic constraints are combined with the equations of motion of the manipulators to obtain a dynamical model of the entire system in the joint space. Reduced-order dynamic equations are then developed with regard to the position and force control variables. Robust control laws are then determined such that the force and position control design is decoupled. The control laws that will be discussed are: a robust position tracking controller that yields an exponentially stable position tracking error result, and a robust force tracking controller that yields adjustable bounds on the force tracking error.     

Multi-object tracking via MHT with multiple information fusion in surveillance video

Tracking multiple objects is critical to automatic video content analysis and virtual reality. The major problem is how to solve data association problem when ambiguous measurements are caused by objects in close proximity. To tackle this problem, we propose a multiple information fusion-based multiple hypotheses tracking algorithm integrated with appearance feature, local motion pattern feature and repulsion–inertia model for multi-object tracking. Appearance model based on HSV–local binary patterns histogram and local motion pattern based on optical flow are adopted to describe objects. A likelihood calculation framework is proposed to incorporate the similarities of appearance, dynamic process and local motion pattern. To consider the changes in appearance and motion pattern over time, we make use of an effective template updating strategy for each object. In addition, a repulsion–inertia model is adopted to explore more useful information from ambiguous detections. Experimental results show that the proposed approach generates better trajectories with less missing objects and identity switches.     

基于差分图象的多运动目标的检测与跟踪

运动目标的检测与跟踪在许多领域有广泛的应用,它是应用视觉研究的焦点之一。     

Part-based motion descriptor image for human action recognition

This paper presents a novel and efficient framework for human action recognition based on modeling the motion of human body-parts. Intuitively, a collective understanding of human body-part movements can lead to better understanding and representation of any human action. In this paper, we propose a generative representation of the motion of human body-parts to learn and classify human actions. The proposed representation combines the advantages of both local and global representations, encoding the relevant motion information as well as being robust to local appearance changes. Our work is motivated by the pictorial structures model and the framework of sparse representations for recognition. Human body-part movements are represented efficiently through quantization in the polar space. The key discrimination within each action is efficiently encoded by sparse representation for classification. The proposed framework is evaluated on both the KTH and the UCF Sport action datasets and results compared against several state-of-the-art methods.     

Template-Based Gait Authentication Through Bayesian Thresholding

While gait recognition is the mapping of a gait sequence to an identity known to the system, gait authentication refers to the problem of identifying whether a given gait sequence belongs to the claimed identity. A typical gait authentication system starts with a feature representation such as a gait template, then proceeds to extract its features, and a transformation is ultimately applied to obtain a discriminant feature set. Almost every authentication approach in literature favours the use of Euclidean distance as a threshold to mark the boundary between a legitimate subject and an impostor. This article proposes a method that uses the posterior probability of a Bayes' classifier in place of the Euclidean distance. The proposed framework is applied to template-based gait feature representations and is evaluated using the standard CASIA-B gait database. Our study experimentally demonstrates that the Bayesian posterior probability performs significantly better than the de facto Euclidean distance approach and the cosine distance which is established in research to be the current state of the art.      

Extracting a diagnostic gait signature

This research addresses the question of the existence of prominent diagnostic signatures for human walking extracted from kinematics gait data. The proposed method is based on transforming the joint motion trajectories using wavelets to extract spatio-temporal features which are then fed as input to a vector quantiser; a self-organising map for classification of walking patterns of individuals with and without pathology. We show that our proposed algorithm is successful in extracting features that successfully discriminate between individuals with and without locomotion impairment.     

Accumulation of object representations utilising interaction of robot action and perception

We introduce a robotic-vision system which is able to extract object representations autonomously utilising a tight interaction of visual perception and robotic action within a perception action cycle [Ecological Psychology 4 (1992) 121; Algebraic Frames for the Perception and Action Cycle, 1997, 1]. Controlled movement of the object grasped by the robot enables us to compute the transformations of entities which are used to represent aspects of objects and to find correspondences of entities within an image sequence.A general accumulation scheme allows to acquire robust information from partly missing information extracted from single frames of an image sequence. Here we use this scheme with a preprocessing stage in which 3D-line segments are extracted from stereo images. However, the accumulation scheme can be used with any kind of preprocessing as long as the entities used to represent objects can be brought to correspondence by certain equivalence relations such as ‘rigid body motion’.We show that an accumulated representation can be applied within a tracking algorithm. The accumulation scheme is an important module of a vision based robot system on which we are currently working. In this system, objects are planned to be represented by different visual and tactile entities. The object representations are going to be learned autonomously. We discuss the accumulation scheme in the context of this project.     

Simultaneous people tracking and motion pattern learning

The field of Human Robot Interaction (HRI) encompasses many difficult challenges as robots need a better understanding of human actions. Human detection and tracking play a major role in such scenarios. One of the main challenges is to track them with long term occlusions due to agile nature of human navigation. However, in general humans do not make random movements. They tend to follow common motion patterns depending on their intentions and environmental/physical constraints. Therefore, knowledge of such common motion patterns could allow a robotic device to robustly track people even with long term occlusions. On the other hand, once a robust tracking is achieved, they can be used to enhance common motion pattern models allowing robots to adapt to new motion patterns that could appear in the environment. Therefore, this paper proposes to learn human motion patterns based on Sampled Hidden Markov Model (SHMM) and simultaneously track people using a particle filter tracker. The proposed simultaneous people tracking and human motion pattern learning has not only improved the tracking robustness compared to more conservative approaches, it has also proven robustness to prolonged occlusions and maintaining identity. Furthermore, the integration of people tracking and on-line SHMM learning have led to improved learning performance. These claims are supported by real world experiments carried out on a robot with suite of sensors including a laser range finder.     

机器人视觉跟踪轨迹的分级规划方法

针对跟踪轨迹规划对于确保得到连续光滑的跟踪运动的重要性,提出了两级视觉跟踪轨迹规划方法。第一阶段在图像平面上规划运动轨迹,在图像平面上得到的离散规划点映射到机器人关节空间。第二阶段在机器人关节空间中用三次样条函数来连接这些离散点。为了满足实时控制的要求,在图像处理过程中采用窗口技术并抽取边缘特征。建立用于跟踪两维平面运动物体(如随运输带运动的物体)的机器人视觉跟踪控制系统。实验结果表明,跟踪误差渐近地减小到允许的数值范围,所提出的跟踪轨迹规划方法是有效的。     

Real-Time Motion Tracking of Robot Manipulators Using Adaptive Genetic Algorithms

The paper presents a genetic algorithm approach to real-time motion tracking of redundant and non-redundant manipulators. The joint angle trajectories are found by applying genetic operators to a set of suitably generated configurations so that the end-effector follows a desired workspace trajectory accurately. The probability of applying a particular genetic operator is adapted on-line to achieve fast convergence to the solution. The adaptation is based on two measures, namely, diversity and fitness of the generated configurations. In order to achieve real time tracking, special provisions are made so that only an appropriate small region in the joint space is searched. The tracking problem is solved at the position level rather the then velocity level. As such the proposed method does not use the manipulator Jacobian inverse or pseudo-inverse matrix and is shown to be free from problems such as excessive joint velocities due to singularities. Simulation results are presented for the 6-DOF Puma and the 7-DOF Robotic Research arm that demonstrate good tracking accuracy and reasonable joint velocities.     

A video-based door monitoring system using local appearance-based face models

In this paper, we present a real-time video-based face recognition system. The developed system identifies subjects while they are entering a room. This application scenario poses many challenges. Continuous, uncontrolled variations of facial appearance due to illumination, pose, expression, and occlusion of non-cooperative subjects need to be handled to allow for successful recognition. In order to achieve this, the system first detects and tracks the eyes for proper registration. The registered faces are then individually classified by a local appearance-based face recognition algorithm. The obtained confidence scores from each classification are progressively combined to provide the identity estimate of the entire sequence. We introduce three different measures to weight the contribution of each individual frame to the overall classification decision. They are distance-to-model (DTM), distance-to-second-closest (DT2ND), and their combination. We have conducted closed-set and open-set identification experiments on a database of 41 subjects. The experimental results show that the proposed system is able to reach high correct recognition rates. Besides, it is able to perform facial feature and face detection, tracking, and recognition in real-time.