Automatic motion detection for surveillance

Automatic motion detection features are able to enhance surveillance efficiency and quality. The aim of this research is to recognize and detect motion automatically around a robot's environment in order to equip a mobile robot for a surveillance task. The required information is based on the input obtained from a charge coupled device (CCD) camera mounted on the mobile robot. As the first step toward achieving the goal, it is necessary to have a stationary mobile robot and moving objects. Experiments in a different environment, such as different movements, size of moving objects, and lighting conditions, have also been conducted. The “adjacent pixels comparison” is the proposed method to detect motion in this experiment. The results have verified that the motion detection experiments operate as expected. This work was presented in part at the 11^th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Hybrid motion graph for character motion synthesis

ObjectiveThis paper proposes a novel framework of Hybrid Motion Graph (HMG) for creating character animations, which enhances the graph-based structural control by motion field representations for efficient motion synthesis of diverse and interactive character animations.MethodsIn HMG framework, the motion template of each class is automatically derived from the training motions for capturing the general spatio-temporal characteristics of an entire motion class. Typical motion field for each class is then constructed. The smooth transitions among motion classes are then generated by interpolating the related motion templates with spacetime constraints. Finally, a hybrid motion graph is built by integrating the separate motion fields for each motion class into the global structural control of motion graph through smooth transition.ResultsIn motion synthesis stage, a character may freely ‘switch’ among different motion classes in the hybrid motion graph via smooth transitions between motion templates and ‘flow’ within each class through the continuous space of motion field with agile and the continuous control process.ConclusionExperimental results show that our framework realizes the fast connectivity among different motion classes and high responsiveness and interactivity for creating realistic character animation of rich behaviors with limited motion data and computational resources.     

Dust particle detection in traffic surveillance video using motion singularity analysis

Dust particle detection in video aims to automatically determine whether the video is degraded by dust particle or not. Dust particles are usually stuck on the camera lends and typically temporally static in the images of a video sequence captured from a dynamic scene. The moving objects in the scene can be occluded by the dusts; consequently, the motion information of moving objects tends to yield singularity. Motivated by this, a dust detection approach is proposed in this paper by exploiting motion singularity analysis in the video. First, the optical model of dust particle is theoretically studied in by simulating optical density of artifacts produced by dust particles. Then, the optical flow is exploited to perform motion singularity analysis for blind dust detection in the video without the need for ground truth dust-free video. More specifically, a singularity model of optical flow is proposed in this paper using the direction of the motion flow field, instead of the amplitude of the motion flow field. The proposed motion singularity model is further incorporated into a temporal voting mechanism to develop an automatic dust particle detection in the video. Experiments are conducted using both artificially-simulated dust-degraded video and real-world dust-degraded video to demonstrate that the proposed approach outperforms conventional approaches to achieve more accurate dust detection.     

The influence of perceptual grouping on motion detection

In this paper, we present a perceptual organization-based method for detecting moving objects from image sequences. To achieve the characteristics of real-time, efficiency, and robustness, a perceptual computation model of edge partitioning and grouping was proposed for the extraction of edge traces on the fly. Each edge trace is made up of generic edge tokens (GETs) which are perceptual features, and defined qualitatively based on the principles of Gestalt laws. Motion detection uses two basic computations: (1) segment motion GETs (MGETs) by computing the gradient differences between GET streams in consecutive frames; and (2) detect motion objects by perceptually grouping MGETs into object clusters. The MGETs in each cluster are constrained by the proximity of the features, and the motion continuation of the cluster measured by motion persistence, etc. Experimental results are provided.     

Segmentation and range sensing using a moving-aperture lens

The use of a novel motorized lens to perform segmentation of image sequences is presented in this paper. The lens has the effect of introducing small, repeating movements of the camera center so that objects appear to translate in the image by an amount that depends on the distance from the plane of focus. For a stationary scene, optical flow magnitudes are therefore directly related to three-dimensional object distance from the observer. We describe a segmentation procedure that exploits these controlled observer movements and present experimental results that demonstrate the successful extraction of objects at different depths. Potential applications of our approach include image compositing, teleconferencing, and range estimation.Received: 4 July 2002, Accepted: 16 December 2002, Published online: 23 July 2003 Correspondence to: Amy E. Bell     

Spatio-temporal structure of human motion primitives and its application to motion prediction

This paper proposes a novel approach to structuring behavioral knowledge based on symbolization of human whole body motions, hierarchical classification of the motions, and extraction of the causality among the motions. The motion patterns are encoded into parameters of corresponding Hidden Markov Models (HMMs), where each HMM abstracts the dynamics of motion pattern, and hereafter is referred to as “motion symbol”. The motion symbols allow motion recognition and synthesis. The motion symbols are organized into a hierarchical tree structure representing the property of spatial similarity among the motion patterns, and this tree is referred to as “motion symbol tree”. Seamless motion is segmented into a sequence of motion primitives, each of which is classified as a motion symbol based on the motion symbol tree. The seamless motion results in a sequence of the motion symbols, which is stochastically represented as transitions between the motion symbols by an N-gram model. The motion symbol N-gram model is referred to as “motion symbol graph”. The motion symbol graph extracts the temporal causality among the human behaviors. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a motion dataset of radio calisthenics and on a large motion dataset provided by CMU motion database validate the proposed framework.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

Learning motion patterns in unstructured scene based on latent structural information

ContextAs trajectory analysis is widely used in the fields of video surveillance, crowd monitoring, behavioral prediction, and anomaly detection, finding motion patterns is a fundamental task for pedestrian trajectory analysis.ObjectiveIn this paper, we focus on learning dominant motion patterns in unstructured scene.MethodsAs the invisible implicit indicator to scene structure, latent structural information is first defined and learned by clustering source/sink points using CURE algorithm. Considering the basic assumption that most pedestrians would find the similar paths to pass through an unstructured scene if their entry and exit areas are fixed, trajectories are then grouped based on the latent structural information. Finally, the motion patterns are learned for each group, which are characterized by a series of statistical temporal and spatial properties including length, duration and envelopes in polar coordinate space.ResultsExperimental results demonstrate the feasibility and effectiveness of our method, and the learned motion patterns can efficiently describe the statistical spatiotemporal models of the typical pedestrian behaviors in a real scene. Based on the learned motion patterns, abnormal or suspicious trajectories are detected.ConclusionThe performance of our approach shows high spatial accuracy and low computational cost.     

A simple algorithm for intelligent manipulator collision-free motion

The collision-free planning of motion is a fundamental problem for artificial intelligence applications in robotics. The ability to compute a continuous safe path for a robot in a given environment will make possible the development of task-level robot planning systems so that the implementation details and the particular robot motion sequence will be ignored by the programmer.A new approach to planning collision-free motions for general real-life six degrees of freedom (d.o.f.) manipulators is presented. It is based on a simple object model previously developed. The complexity of the general collision detection problem is reduced, and realistic collision-free paths are efficiently found onCS planes. A heuristic evaluation function with a real physical sense is introduced, and computational cost is reduced to the strictly necessary by selecting the most adequate level of representation. A general algorithm is defined for 6 d.o.f. robots that yields good results for actual robot models with complex design structures with the aid of various heuristic techniques. The problem of adaptive motion is also considered.     

Semantic Plane-Structure based motion detection with a nonstationary camera

This paper presents a novel method to accurately detect moving objects from a video sequence captured using a nonstationary camera. Although common methods provide effective motion detection for static backgrounds or through only planar-perspective transformation, many detection errors occur when the background contains complex dynamic interferences or the camera undergoes unknown motions. To solve this problem, this study proposed a motion detection method that incorporates temporal motion and spatial structure. In the proposed method, first, spatial semantic planes are segmented, and image registration based on stable background planes is applied to overcome the interferences of the foreground and dynamic background. Thus, the estimated dense temporal motion ensures that small moving objects are not missed. Second, motion pixels are mapped on semantic planes, and then, the spatial distribution constraints of motion pixels, regional shapes and plane semantics, which are integrated into a planar structure, are used to minimise false positives. Finally, based on the dense temporal motion and spatial structure, moving objects are accurately detected. The experimental results on CDnet dataset, Pbi dataset, Aeroscapes dataset, and other challenging self-captured videos under difficult conditions, such as fast camera movement, large zoom variation, video jitters, and dynamic background, revealed that the proposed method can remove background movements, dynamic interferences, and marginal noises and can effectively obtain complete moving objects.© 2017 ElsevierInc.Allrightsreserved.     

Human motions representation and recognition by directional motion history images

In this paper, we describe a technique for representing and recognizing human motions using directional motion history images. A motion history image is a single human motion image produced by superposing binarized successive motion image frames so that older frames may have smaller weights. It has, however, difficulty that the latest motion overwrites older motions, resulting in inexact motion representation and therefore incorrect recognition. To overcome this difficulty, we propose directional motion history images which describe a motion with respect to four directions of movement, i.e. up, down, right and left, employing optical flow. The directional motion history images are thus a set of four motion history images defined on four optical flow images. Experimental results show that the proposed technique achieves better performance in the recognition of human motions than the existent motion history images. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

A real-time system for motion retrieval and interpretation

This paper proposes a new examplar-based method for real-time human motion recognition using Motion Capture (MoCap) data. We have formalized streamed recognizable actions, coming from an online MoCap engine, into a motion graph that is similar to an animation motion graph. This graph is used as an automaton to recognize known actions as well as to add new ones. We have defined and used a spatio-temporal metric for similarity measurements to achieve more accurate feedbacks on classification. The proposed method has the advantage of being linear and incremental, making the recognition process very fast and the addition of a new action straightforward. Furthermore, actions can be recognized with a score even before they are fully completed. Thanks to the use of a skeleton-centric coordinate system, our recognition method has become view-invariant. We have successfully tested our action recognition method on both synthetic and real data. We have also compared our results with four state-of-the-art methods using three well known datasets for human action recognition. In particular, the comparisons have clearly shown the advantage of our method through better recognition rates.     

People tracking based on motion model and motion constraints with automatic initialization

Human motion analysis is currently one of the most active research topics in computer vision. This paper presents a model-based approach to recovering motion parameters of walking people from monocular image sequences in a CONDENSATION framework. From the semi-automatically acquired training data, we learn a motion model represented as Gaussian distributions, and explore motion constraints by considering the dependency of motion parameters and represent them as conditional distributions. Then both of them are integrated into a dynamic model to concentrate factored sampling in the areas of the state-space with most posterior information. To measure the observation density with accuracy and robustness, a pose evaluation function (PEF) combining both boundary and region information is proposed. The function is modeled with a radial term to improve the efficiency of the factored sampling. We also address the issue of automatic acquisition of initial model pose and recovery from severe failures. A large number of experiments carried out in both indoor and outdoor scenes demonstrate that the proposed approach works well     

Content adaptive fast motion estimation based on spatio-temporal homogeneity analysis and motion classification

In video coding, research is focused on the development of fast motion estimation (ME) algorithms while keeping the coding distortion as small as possible. It has been observed that the real world video sequences exhibit a wide range of motion content, from uniform to random, therefore if the motion characteristics of video sequences are taken into account before hand, it is possible to develop a robust motion estimation algorithm that is suitable for all kinds of video sequences. This is the basis of the proposed algorithm. The proposed algorithm involves a multistage approach that includes motion vector prediction and motion classification using the characteristics of video sequences. In the first step, spatio-temporal correlation has been used for initial search centre prediction. This strategy decreases the effect of unimodal error surface assumption and it also moves the search closer to the global minimum hence increasing the computation speed. Secondly, the homogeneity analysis helps to identify smooth and random motion. Thirdly, global minimum prediction based on unimodal error surface assumption helps to identify the proximity of global minimum. Fourthly, adaptive search pattern selection takes into account various types of motion content by dynamically switching between stationary, center biased and, uniform search patterns. Finally, the early termination of the search process is adaptive and is based on the homogeneity between the neighboring blocks.Extensive simulation results for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality and less complexity as compared to other ME algorithms. Implementation of proposed algorithm in JM12.2 of H.264/AVC shows reduction in computational complexity measured in terms of encoding time while maintaining almost same bit rate and PSNR as compared to Full Search algorithm.     

Polygon-based post-process motion blur

This paper discusses a three-pass raster motion-blur algorithm (and some generalizations) in the context of texture-mapped polygons and its application to blurring objects and surfaces made up of multiple polygons, which may move in different directions.     

A boundary control for motion synchronization of a two-manipulator system with a flexible beam

A novel synchronization motion control method is proposed in this paper for the system in which two manipulators are constrained by a flexible beam. Different from the general synchronization control method, the coupling dynamics among various actuators is considered as the shear force, which results from the synchronization errors. Then a simple boundary control is introduced to realize the synchronization motion of actuators by suppressing the shear force. In order to avoid the drawbacks of assumed modes model, the dynamic model of flexible beam is described by a distributed parameter model in this paper. A Riesz basis method is used to prove that the proposed control law can guarantee the synchronization system to be exponential stability. Simulation results demonstrate that the proposed method can effectively improve the performance of synchronization motion compared with other methods.     

Self-indexed motion planning

Motion planning is a central problem for robotics. A practical way to address it is building a graph-based representation (a roadmap) capturing the connectivity of the configuration space. The Probabilistic Road Map (PRM) is perhaps the most widely used method by the robotics community based on that idea. A key sub-problem for discovering and maintaining a collision-free path in the PRM is inserting new sample points and connecting them with the $k$-nearest neighbors in the previous set. Instead of following the usual solution of indexing the points and then building the PRM with successive $k$-NN queries, we propose an approximation of the $k$-Nearest Neighbors Graph using the PRM as a self-index. The motivation for this construction comes from the Approximate Proximity Graph (APG), which is an index for searching proximal objects in a metric space. Using this approach the estimation of the $k$-NN is improved while simultaneously reducing the total time and space needed to compute a PRM. We present simulations for high-dimensional configuration spaces with and without obstacles, showing significant improvement over the standard techniques used by the robotics community.     

Object-oriented motion abstraction

An important problem in the production of an animation sequence is the great amount of information necessary to control and specify the motion. Specification of complex animation sequences with a smaller amount of information is possible if they are built over some abstracted sequences. Abstraction supports dealing with complexity by structuring, so that the necessary features are made available while those that are not necessary are hidden. In our work, motion abstraction is used to build complex animation sequences with the help of object-oriented concepts. A parametric key-frame interpolation method is used for producing the in-between frames of an animation sequence. The parameters that define the motion of a model, in our work, are position, orientation, size, shape and colour. Orientation transformations are implemented by unit quaternions.     

Real-time crowd motion planning

Real-time crowd motion planning requires fast, realistic methods for path planning as well as obstacle avoidance. In a previous work (Morini et al. in Cyberworlds International Conference, pp. 144–151, 2007), we introduced a hybrid architecture to handle real-time motion planning of thousands of pedestrians. In this article, we present an extended version of our architecture, introducing two new features: an improved short-term collision avoidance algorithm, and simple efficient group behavior for crowds. Our approach allows the use of several motion planning algorithms of different precision for regions of varied interest. Pedestrian motion continuity is ensured when switching between such algorithms. To assess our architecture, several performance tests have been conducted, as well as a subjective test demonstrating the impact of using groups. Our results show that the architecture can plan motion in real time for several thousands of characters.