Adaptive guaranteed estimation based on information provided by a set of “Relative” objects

Estimation algorithms for the state of an object based on “relative” objects information are proposed, under conditions of disturbances affecting the object and errors caused by incomplete and inaccurate measurements of the object’s vector. Information on disturbances and measurement errors is considered known up to some given sets. If the path of the object’s motion is changed, it is suggested to use information provided by a group of similar objects for identifying model of the object’s further motion.     

A novel unsupervised 3D skeleton detection in RGB-D images for video surveillance

In this paper we present a novel moment-based skeleton detection for representing human objects in RGB-D videos with animated 3D skeletons. An object often consists of several parts, where each of them can be concisely represented with a skeleton. However, it remains as a challenge to detect the skeletons of individual objects in an image since it requires an effective part detector and a part merging algorithm to group parts into objects. In this paper, we present a novel fully unsupervised learning framework to detect the skeletons of human objects in a RGB-D video. The skeleton modeling algorithm uses a pipeline architecture which consists of a series of cascaded operations, i.e., symmetry patch detection, linear time search of symmetry patch pairs, part and symmetry detection, symmetry graph partitioning, and object segmentation. The properties of geometric moment-based functions for embedding symmetry features into centers of symmetry patches are also investigated in detail. As compared with the state-of-the-art deep learning approaches for skeleton detection, the proposed approach does not require tedious human labeling work on training images to locate the skeleton pixels and their associated scale information. Although our algorithm can detect parts and objects simultaneously, a pre-learned convolution neural network (CNN) can be used to locate the human object from each frame of the input video RGB-D video in order to achieve the goal of constructing real-time applications. This much reduces the complexity to detect the skeleton structure of individual human objects with our proposed method. Using the segmented human object skeleton model, a video surveillance application is constructed to verify the effectiveness of the approach. Experimental results show that the proposed method gives good performance in terms of detection and recognition using publicly available datasets.

     

A color-based particle filter for multiple object tracking in an outdoor environment

Tracking multiple objects is more challenging than tracking a single object. Some problems arise in multiple-object tracking that do not exist in single-object tracking, such as object occlusion, the appearance of a new object and the disappearance of an existing object, updating the occluded object, etc. In this article, we present an approach to handling multiple-object tracking in the presence of occlusions, background clutter, and changing appearance. The occlusion is handled by considering the predicted trajectories of the objects based on a dynamic model and likelihood measures. We also propose target-model-update conditions, ensuring the proper tracking of multiple objects. The proposed method is implemented in a probabilistic framework such as a particle filter in conjunction with a color feature. The particle filter has proven very successful for nonlinear and non-Gaussian estimation problems. It approximates a posterior probability density of the state, such as the object’s position, by using samples or particles, where each state is denoted as the hypothetical state of the tracked object and its weight. The observation likelihood of the objects is modeled based on a color histogram. The sample weight is measured based on the Bhattacharya coefficient, which measures the similarity between each sample’s histogram and a specified target model. The algorithm can successfully track multiple objects in the presence of occlusion and noise. Experimental results show the effectiveness of our method in tracking multiple objects.     

Object motion tracking using a moving direction estimate and color updates

This paper presents a direction detection and tracking object color update algorithm used to track moving objects that change colors. Different from traditional color-based tracking methods, which use an initial color distribution in order to track objects as long as the object carries the full or partial initial color, this method introduces a color update method used to quickly find the new object color in a new location if the object changes its color partially or completely; the updated color is then used to locate the object. In our algorithm, an initial color pattern is used to track an object using the color. During the tracking, an object’s new location is at first estimated and then used to detect any color change. If the color has changed, a new color pattern is updated based on the changes in the previous color distribution, and then the new color pattern is used to calculate the current location of the object. This algorithm utilizes the property that the movement of an object can be estimated either by using the object’s shadow or by background subtraction. The implementation of our algorithm results in an effective real-time object tracking. The validity of the approach is illustrated by the presentation of experiment results obtained using the methods described in this paper.     

Modeling and synthesis of supervisory control based on Petri nets for distributed objects. I. Interaction mechanism and the basic method

We describe a methodology of discrete event modeling for a class of distributed objects and their required behavior (specifications) for the design of real time automation systems. In our methodology, we use the structured discrete event system (SDES2) model: on the first stage, it is used to analyze the functionality and coherence of the object and its specification; on the second, we propose for SDES2 a basic synthesis method that works for the models of object and supervisor based on Petri nets (both modeling and controlling). At the same time, we propose to synthesize the supervisor as a Petri net embedded in SDES2 with a feedback circuit in order to restrict the object’s operation according to specification requirements. We propose an interaction mechanism for the modeling and controlling Petri nets with the object and the external environment. In essence, the interaction mechanism is an object control scheme based on the constructed net. This mechanism analyzes the current state of the object and computes the controls that should be passed on to the object’s actuators. Computations are done with a cyclic procedure looping over the matrix representation of the net.     

基于非脊点下降算子的多尺度骨架化算法

骨架是目标表示的一种重要方式.提出了一种基于区域标记直接从灰度图像中提取的骨架的新算法.算法对脊点概念作了补充撰述,组合利用了目标的轮廓与区域信息,采用了层次化的处理策略,适用于稳健地提取规则和不规则目标完整的多尺度骨架.所提取的骨架彼此连通、单像素宽并与原始图像拓扑一致.将算法应用于实际图像,检测到了与人视觉感知相一致的目标骨架.     

Path similarity skeleton graph matching

This paper presents a novel framework to for shape recognition based on object silhouettes. The main idea is to match skeleton graphs by comparing the shortest paths between skeleton endpoints. In contrast to typical tree or graph matching methods, we completely ignore the topological graph structure. Our approach is motivated by the fact that visually similar skeleton graphs may have completely different topological structures. The proposed comparison of shortest paths between endpoints of skeleton graphs yields correct matching results in such cases. The skeletons are pruned by contour partitioning with Discrete Curve Evolution, which implies that the endpoints of skeleton branches correspond to visual parts of the objects. The experimental results demonstrate that our method is able to produce correct results in the presence of articulations, stretching, and occlusion.     

3D Reconstruction by Shadow Carving: Theory and Practical Evaluation

Cast shadows are an informative cue to the shape of objects. They are particularly valuable for discovering object’s concavities which are not available from other cues such as occluding boundaries. We propose a new method for recovering shape from shadows which we call shadow carving. Given a conservative estimate of the volume occupied by an object, it is possible to identify and carve away regions of this volume that are inconsistent with the observed pattern of shadows. We prove a theorem that guarantees that when these regions are carved away from the shape, the shape still remains conservative. Shadow carving overcomes limitations of previous studies on shape from shadows because it is robust with respect to errors in shadows detection and it allows the reconstruction of objects in the round, rather than just bas-reliefs. We propose a reconstruction system to recover shape from silhouettes and shadow carving. The silhouettes are used to reconstruct the initial conservative estimate of the object’s shape and shadow carving is used to carve out the concavities. We have simulated our reconstruction system with a commercial rendering package to explore the design parameters and assess the accuracy of the reconstruction. We have also implemented our reconstruction scheme in a table-top system and present the results of scanning of several objects.     

Browsing the environment with the SNAP&TELL wearable computer system

This paper provides an overview of a multi-modal wearable computer system, SNAP&TELL. The system performs real-time gesture tracking, combined with audio-based control commands, in order to recognize objects in an environment, including outdoor landmarks. The system uses a single camera to capture images, which are then processed to perform color segmentation, fingertip shape analysis, robust tracking, and invariant object recognition, in order to quickly identify the objects encircled and SNAPped by the user’s pointing gesture. In addition, the system returns an audio narration, TELLing the user information concerning the object’s classification, historical facts, usage, etc. This system provides enabling technology for the design of intelligent assistants to support “Web-On-The-World” applications, with potential uses such as travel assistance, business advertisement, the design of smart living and working spaces, and pervasive wireless services and internet vehicles. An erratum to this article can be found at An erratum to this article can be found at     

Computing multiscale curve and surface skeletons of genus 0 shapes using a global importance measure

We present a practical algorithm for computing robust, multiscale curve and surface skeletons of 3D objects. Based on a model which follows an advection principle, we assign to each point on the skeleton a part of the object surface, called the collapse. The size of the collapse is used as a uniform importance measure for the curve and surface skeleton, so that both can be simplified by imposing a single threshold on this intuitive measure. The simplified skeletons are connected by default, without special precautions, due to the monotonicity of the importance measure. The skeletons possess additional desirable properties: They are centered, robust to noise, hierarchical, and provide a natural skeleton-to-boundary mapping. We present a voxel-based algorithm that is straightforward to implement and simple to use. We illustrate our method on several realistic 3D objects.     

Localization of a mobile robot based on an ultrasonic sensor using dynamic obstacles

In this article, we propose a localization scheme for a mobile robot based on the distance between the robot and moving objects. This method combines the distance data obtained from ultrasonic sensors in a mobile robot, and estimates the location of the mobile robot and the moving object. The movement of the object is detected by a combination of data and the object’s estimated position. Then, the mobile robot’s location is derived from the a priori known initial state. We use kinematic modeling that represents the movement of a robot and an object. A Kalman-filtering algorithm is used for addressing estimation error and measurement noise. Throughout the computer simulation experiments, the performance is verified. Finally, the results of experiments are presented and discussed. The proposed approach allows a mobile robot to seek its own position in a weakly structured environment. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

Object Level Grouping for Video Shots

We describe a method for automatically obtaining object representations suitable for retrieval from generic video shots. The object representation consists of an association of frame regions. These regions provide exemplars of the object’s possible visual appearances. Two ideas are developed: (i) associating regions within a single shot to represent a deforming object; (ii) associating regions from the multiple visual aspects of a 3D object, thereby implicitly representing 3D structure. For the association we exploit temporal continuity (tracking) and wide baseline matching of affine covariant regions. In the implementation there are three areas of novelty: First, we describe a method to repair short gaps in tracks. Second, we show how to join tracks across occlusions (where many tracks terminate simultaneously). Third, we develop an affine factorization method that copes with motion degeneracy. We obtain tracks that last throughout the shot, without requiring a 3D reconstruction. The factorization method is used to associate tracks into object-level groups, with common motion. The outcome is that separate parts of an object that are not simultaneously visible (such as the front and back of a car, or the front and side of a face) are associated together. In turn this enables object-level matching and recognition throughout a video. We illustrate the method on the feature film “Groundhog Day.” Examples are given for the retrieval of deforming objects (heads, walking people) and rigid objects (vehicles, locations).     

Efficient real-time trajectory tracking

Moving objects databases (MOD) manage trajectory information of vehicles, animals, and other mobile objects. A crucial problem is how to efficiently track an object’s trajectory in real-time, in particular if the trajectory data is sensed at the mobile object and thus has to be communicated over a wireless network. We propose a family of tracking protocols that allow trading the communication cost and the amount of trajectory data stored at a MOD off against the spatial accuracy. With each of these protocols, the MOD manages a simplified trajectory that does not deviate by more than a certain accuracy bound from the actual movement. Moreover, the different protocols enable several trade-offs between computational costs, communication cost, and the reduction in the trajectory data: Connection-Preserving Dead Reckoning minimizes the communication cost using dead reckoning, a technique originally designed for tracking an object’s current position. Generic Remote Trajectory Simplification (GRTS) further separates between tracking of the current position and simplification of the past trajectory and can be realized with different line simplification algorithms. For both protocols, we discuss how to bound the space consumption and computing time at the moving object and thereby present an effective compression technique to optimize the reduction performance of real-time line simplification in general. Our evaluations with hundreds of real GPS traces show that a realization of GRTS with a simple simplification heuristic reaches 85–90% of the best possible reduction rate, given by retrospective offline simplification. A realization with the optimal line simplification algorithm by Imai and Iri even reaches more than 97% of the best possible reduction rate.     

Skeleton pruning by contour partitioning with discrete curve evolution

In this paper, we introduce a new skeleton pruning method based on contour partitioning. Any contour partition can be used, but the partitions obtained by discrete curve evolution (DCE) yield excellent results. The theoretical properties and the experiments presented demonstrate that obtained skeletons are in accord with human visual perception and stable, even in the presence of significant noise and shape variations, and have the same topology as the original skeletons. In particular, we have proven that the proposed approach never produces spurious branches, which are common when using the known skeleton pruning methods. Moreover, the proposed pruning method does not displace the skeleton points. Consequently, all skeleton points are centers of maximal disks. Again, many existing methods displace skeleton points in order to produces pruned skeletons     

Effective skeletons extraction for animated surfaces based on geometry propagation

We introduce a registration‐based propagation strategy to extract temporally coherent skeletons from the animated surfaces. We first extract complete skeletons for the key frames of the animated surfaces, and present a spatio‐temporal L₁‐medial skeleton extraction method to extract the initial skeletons for the immediate frames between the key frames. As these initial skeletons may not be complete, we then develop a global skeleton alignment method, which effectively warps the key skeleton to these initial skeletons subsequently. By using the geometry merging, we propagate the geometry of the warped key skeleton to the initial skeletons, and produce temporally coherent skeletons for the input animated surfaces. Our system can be applied to the raw scanned animated object and requires only minimal user interaction to extract complete and temporally coherent animated skeletons. Copyright © 2015 John Wiley & Sons, Ltd.     

Hierarchical part-type segmentation using voxel-based curve skeletons

We present an effective framework for segmenting 3D shapes into meaningful components using the curve skeleton. Our algorithm identifies a number of critical points on the efficiently computed curve skeleton, either fully automatically as the junctions of the curve skeleton, or based on user input. We use these points to construct a partitioning of the object surface using geodesics. Because the segmentation is based on the curve skeleton, it intrinsically reflects the shape symmetry and articulation, and can handle shapes with tunnels. We describe a voxel-based implementation of our method which is robust and noise resistant, able to handle shapes of complex articulation and topology, produces smooth segment borders, and delivers hierarchical level-of-detail segmentations. We demonstrate the framework on various real-world 3D shapes. Additionally, we discuss the use of both curve and surface skeletons to produce part-type and patch-type, respectively, segmentations of 3D shapes.     

Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking

In this article we present the integration of 3-D shape knowledge into a variational model for level set based image segmentation and contour based 3-D pose tracking. Given the surface model of an object that is visible in the image of one or multiple cameras calibrated to the same world coordinate system, the object contour extracted by the segmentation method is applied to estimate the 3-D pose parameters of the object. Vice-versa, the surface model projected to the image plane helps in a top-down manner to improve the extraction of the contour. While common alternative segmentation approaches, which integrate 2-D shape knowledge, face the problem that an object can look very differently from various viewpoints, a 3-D free form model ensures that for each view the model can fit the data in the image very well. Moreover, one additionally solves the problem of determining the object’s pose in 3-D space. The performance is demonstrated by numerous experiments with a monocular and a stereo camera system.