Computing occluding and transparent motions

Computing the motions of several moving objects in image sequences involves simultaneous motion analysis and segmentation. This task can become complicated when image motion changes significantly between frames, as with camera vibrations. Such vibrations make tracking in longer sequences harder, as temporal motion constancy cannot be assumed. The problem becomes even more difficult in the case of transparent motions.A method is presented for detecting and tracking occluding and transparent moving objects, which uses temporal integration without assuming motion constancy. Each new frame in the sequence is compared to a dynamic internal representation image of the tracked object. The internal representation image is constructed by temporally integrating frames after registration based on the motion computation. The temporal integration maintains sharpness of the tracked object, while blurring objects that have other motions. Comparing new frames to the internal representation image causes the motion analysis algorithm to continue tracking the same object in subsequent frames, and to improve the segmentation.     

Surveillance and human–computer interaction applications of self-growing models

The aim of the work is to build self-growing based architectures to support visual surveillance and human–computer interaction systems. The objectives include: identifying and tracking persons or objects in the scene or the interpretation of user gestures for interaction with services, devices and systems implemented in the digital home. The system must address multiple vision tasks of various levels such as segmentation, representation or characterization, analysis and monitoring of the movement to allow the construction of a robust representation of their environment and interpret the elements of the scene.It is also necessary to integrate the vision module into a global system that operates in a complex environment by receiving images from acquisition devices at video frequency and offering results to higher level systems, monitors and take decisions in real time, and must accomplish a set of requirements such as: time constraints, high availability, robustness, high processing speed and re-configurability.Based on our previous work with neural models to represent objects, in particular the Growing Neural Gas (GNG) model and the study of the topology preservation as a function of the parameters election, it is proposed to extend the capabilities of this self-growing model to track objects and represent their motion in image sequences under temporal restrictions.These neural models have various interesting features such as: their ability to readjust to new input patterns without restarting the learning process, adaptability to represent deformable objects and even objects that are divided in different parts or the intrinsic resolution of the problem of matching features for the sequence analysis and monitoring of the movement. It is proposed to build an architecture based on the GNG that has been called GNG-Seq to represent and analyze the motion in image sequences. Several experiments are presented that demonstrate the validity of the architecture to solve problems of target tracking, motion analysis or human–computer interaction.     

A motion observable representation using color correlogram and its applications to tracking

This paper presents a special form of color correlogram as representation for object tracking and carries out a motion observability analysis to obtain the optimal correlogram in a kernel based tracking framework. Compared with the color histogram, where the position information of each pixel is ignored, a simplified color correlogram (SCC) representation encodes the spatial information explicitly and enables an estimation algorithm to recover the object orientation. In this paper, based on the SCC representation, the mean shift algorithm is developed in a translation–rotation joint domain to track the positions and orientations of objects. The ability of the SCC in detecting and estimating object motion is analyzed and a principled way to obtain the optimal SCC as object representation is proposed to ensure reliable tracking. Extensive experimental results demonstrate SCC as a viable object representation for tracking.     

Robot-vision architecture for real-time 6-DOF object localization

This paper presents a new robot-vision system architecture for real-time moving object localization. The 6-DOF (3 translation and 3 rotation) motion of the objects is detected and tracked accurately in clutter using a model-based approach without information of the objects’ initial positions. An object identification task and an object tracking task are combined under this architecture. The computational time-lag between the two tasks is absorbed by a large amount of frame memory. The tasks are implemented as independent software modules using stereo-vision-based methods which can deal with objects of various shapes with edges, including planar to smooth-curved objects, in cluttered environments. This architecture also leads to failure-recoverable object tracking, because the tracking processes can be automatically recovered, even if the moving objects are lost while tracking. Experimental results obtained with prototype systems demonstrate the effectiveness of the proposed architecture.     

Real-time tracking of multiple objects in space-variant vision based on magnocellular visual pathway

In this paper, we propose a space-variant image representation model based on properties of magnocellular visual pathway, which perform motion analysis, in human retina. Then, we present an algorithm for the tracking of multiple objects in the proposed space-variant model. The proposed space-variant model has two effective image representations for object recognition and motion analysis, respectively. Each image representation is based on properties of two types of ganglion cell, which are the beginning of two basic visual pathways; one is parvocellular and the other is magnocellular. Through this model, we can get the efficient data reduction capability with no great loss of important information. And, the proposed multiple objects tracking method is restricted in space-variant image. Typically, an object-tracking algorithm consists of several processes such as detection, prediction, matching, and updating. In particular, the matching process plays an important role in multiple objects tracking. In traditional vision, the matching process is simple when the target objects are rigid. In space-variant vision, however, it is very complicated although the target is rigid, because there may be deformation of an object region in the space-variant coordinate system when the target moves to another position. Therefore, we propose a deformation formula in order to solve the matching problem in space-variant vision. By solving this problem, we can efficiently implement multiple objects tracking in space-variant vision.     

利用视觉显著性和粒子滤波的运动目标跟踪

针对运动目标跟踪问题,提出一种利用视觉显著性和粒子滤波的目标跟踪算法.借鉴人类视觉注意机制的研究成果,根据目标的颜色、亮度和运动等特征形成目标的视觉显著性特征,与目标的颜色分布模型一起作为目标的特征表示模型,利用粒子滤波进行目标跟踪.该算法能够克服利用单一颜色特征所带来的跟踪不稳定问题,并能有效解决由于目标形变、光照变化以及目标和背景颜色分布相似而产生的跟踪困难问题,具有较强的鲁棒性.在多个视频序列中进行实验,并给出相应的实验结果和分析.实验结果表明,该算法用于实现运动目标跟踪是正确有效的.     

Reliable Distributed Network Management by Replication

This paper presents a new clustering architecture for SNMP agents that supports semi-active replication of managed objects. A cluster of agents provides fault-tolerant object functionality: replicated managed objects of a crashed agent of a given cluster may be accessed through a peer cluster. The proposed architecture is structured in three layers. The lower layer corresponds to the managed objects at the network elements. The middle layer contains management entities called clusters that monitor and replicate managed objects. The upper layer allows the definition of management clusters as well as the relationship between clusters. A practical tool was implemented and is presented. The impact of replication on network performance is evaluated as well as a probabilistic analysis of replicated object consistency.     

Predicting Object Dynamics From Visual Images Through Active Sensing Experiences

Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.     

区域损失函数的孪生网络目标跟踪

针对预训练卷积神经网络提取的深度特征空间分辨率低,快速运动造成运动目标空间细节信息丢失等问题,提出用区域损失函数构建孪生网络的目标跟踪,进一步降低深度特征通道之间的冗余性,并减少高层信息丢失。利用线下预训练的VGG-16卷积神经网络提取深度特征,构成初始深度特征空间。通过区域损失函数构建特征和尺度选择网络,根据反向传播的梯度大小进行特征选择。对筛选后的特征进行拼接,融入到孪生网络中匹配跟踪。在OTB-2013、OTB-2015、VOT2016、TempleColor数据集上与其他算法对比。实验结果表明,该算法在快速运动、低分辨率等场景中表现出较好的跟踪精度和鲁棒性。     

Tracking multiple moving objects by binary object forest segmentation

In an earlier study it was shown that the low level image segmentation technique known as binary object forest (BOF) analysis could be successfully used to extract one or two moving objects from complex backgrounds, even when the motion involved was very large. The method involved performing BOF analysis on each of a pair of images from a sequence and then matching the vertices of the resulting graphs. In the present study the problem of tracking multiple objects in complex backgrounds and in difficult circumstances such as partial occlusion, is considered. The approach taken is once again to perform an initial BOF analysis of each image but now to attempt matching over subgraphs of the BOF rather than simply on individual vertices. It is shown theoretically and experimentally that this results in a much more robust matching scheme. This increase in robustness not only allows multiple objects to be tracked but facilitates correct matching even when partial object occlusion occurs and when motion towards the sensor results in large (apparent) size changes between frames.     

A Competitive Neural Network for Multiple Object Tracking in Video Sequence Analysis

Tracking of moving objects in real situation is a challenging research issue, due to dynamic changes in objects or background appearance, illumination, shape and occlusions. In this paper, we deal with these difficulties by incorporating an adaptive feature weighting mechanism to the proposed growing competitive neural network for multiple objects tracking. The neural network takes advantage of the most relevant object features (information provided by the proposed adaptive feature weighting mechanism) in order to estimate the trajectories of the moving objects. The feature selection mechanism is based on a genetic algorithm, and the tracking algorithm is based on a growing competitive neural network where each unit is associated to each object in the scene. The proposed methods (object tracking and feature selection mechanism) are applied to detect the trajectories of moving vehicles in roads. Experimental results show the performance of the proposed system compared to the standard Kalman filter.     

DISCOV: A Framework for Discovering Objects in Video

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.     

Rigid body segmentation and shape description from dense opticalflow under weak perspective

We present an algorithm for identifying and tracking independently moving rigid objects from optical flow. Some previous attempts at segmentation via optical flow have focused on finding discontinuities in the flow field. While discontinuities do indicate a change in scene depth, they do not in general signal a boundary between two separate objects. The proposed method uses the fact that each independently moving object has a unique epipolar constraint associated with its motion. Thus motion discontinuities based on self-occlusion can be distinguished from those due to separate objects. The use of epipolar geometry allows for the determination of individual motion parameters for each object as well as the recovery of relative depth for each point on the object. The algorithm assumes an affine camera where perspective effects are limited to changes in overall scale. No camera calibration parameters are required. A Kalman filter based approach is used for tracking motion parameters with time     

Visual object tracking based on adaptive Siamese and motion estimation network

Recently, convolutional neural network (CNN) has attracted much attention in different areas of computer vision, due to its powerful abstract feature representation. Visual object tracking is one of the interesting and important areas in computer vision that achieves remarkable improvements in recent years. In this work, we aim to improve both the motion and observation models in visual object tracking by leveraging representation power of CNNs. To this end, a motion estimation network (named MEN) is utilized to seek the most likely locations of the target and prepare a further clue in addition to the previous target position. Hence the motion estimation would be enhanced by generating a small number of candidates near two plausible positions. The generated candidates are then fed into a trained Siamese network to detect the most probable candidate. Each candidate is compared to an adaptable buffer, which is updated under a predefined condition. To take into account the target appearance changes, a weighting CNN (called WCNN) adaptively assigns weights to the final similarity scores of the Siamese network using sequence-specific information. Evaluation results on well-known benchmark datasets (OTB100, OTB50 and OTB2013) prove that the proposed tracker outperforms the state-of-the-art competitors.     

Adaptive neural controller for cooperative multiple robot manipulator system manipulating a single rigid object

In this article, an adaptive neural controller is developed for cooperative multiple robot manipulator system carrying and manipulating a common rigid object. In coordinated manipulation of a single object using multiple robot manipulators simultaneous control of the object motion and the internal force exerted by manipulators on the object is required. Firstly, an integrated dynamic model of the manipulators and the object is derived in terms of object position and orientation as the states of the derived model. Based on this model, a controller is proposed that achieves required trajectory tracking of the object as well as tracking of the desired internal forces arising in the system. A feedforward neural network is employed to learn the unknown dynamics of robot manipulators and the object. It is shown that the neural network can cope with the unknown nonlinearities through the adaptive learning process and requires no preliminary offline learning. The adaptive learning algorithm is derived from Lyapunov stability analysis so that both error convergence and tracking stability are guaranteed in the closed loop system. Finally, simulation studies and analysis are carried out for two three-link planar manipulators moving a circular disc on specified trajectory.     

协同运动状态估计的多目标跟踪算法

多目标跟踪在视频分析场景中有着广泛的应用,如人机交互、虚拟现实、自动驾驶、视频监控和机器人导航等。多目标跟踪问题可以表示为在已有的检测数据上进行目标轨迹关联,检测算法的准确性对跟踪性能起着关键性的作用。在基于检测的目标跟踪框架中,提出了一种协同运动状态估计的跟踪算法,该算法主要关注相邻帧之间的数据关联,从目标检测、目标运动状态估计和数据关联这3个方面来直接解决多目标跟踪面临的挑战。首先,对于目标检测,采用Multi Scale Convolutional Neural Network(MS-CNN)算法作为检测器,这是因为深度学习在检测的效益上优于传统的机器学习方法;其次,为了更好地预测目标的运动状态和处理目标间的遮挡,针对不同状态的目标采取不同的运动估计方法: 采用核相关滤波来评估处于跟踪状态的目标的运动状态,当目标处于遮挡状态时,采用卡尔曼滤波做运动估计;最后,采用Kuhn-Munkres算法对检测目标和跟踪轨迹做数据关联。通过大量的实验证实了算法的有效性,且实验结果表明算法的准确性很高。     

Improving the robustness of parametric shape tracking with switched multiple models

This paper addresses the problem of tracking objects with complex motion dynamics or shape changes. It is assumed that some of the visual features detected in the image (e.g., edge strokes) are outliers i.e., they do not belong to the object boundary. A robust tracking algorithm is proposed which allows to efficiently track an object with complex shape or motion changes in clutter environments. The algorithm relies on the use of multiple models, i.e., a bank of stochastic motion models switched according to a probabilistic mechanism. Robust filtering methods are used to estimate the label of the active model as well as the state trajectory.     

A modular neural network for super-resolution of human faces

This paper presents the original and versatile architecture of a modular neural network and its application to super-resolution. Each module is a small multilayer perceptron, trained with the Levenberg-Marquardt method, and is used as a generic building block. By connecting the modules together to establish a composition of their individual mappings, we elaborate a lattice of modules that implements full connectivity between the pixels of the low-resolution input image and those of the higher-resolution output image. After the network is trained with patterns made up of low and high-resolution images of objects or scenes of the same kind, it will be able to enhance dramatically the resolution of a similar object’s representation. The modular nature of the architecture allows the training phase to be readily parallelized on a network of PCs. Finally, it is shown that the network performs global-scale reconstruction of human faces from very low resolution input images.     

Hyperpatches for 3D model acquisition and tracking

Automatic 3D model acquisition and 3D tracking of simple objects under motion using a single camera is often difficult due to the sparsity of information from which to establish the model. We developed an automatic scheme that first computes a simple Euclidean model of the object and then enriches this model using hyperpatches. These hyperpatches contain information on both the orientation and intensity pattern variation of roughly planar patches on an object. This information allows both the spatial and intensity distortions of the projected patch to be modeled accurately under 3D object motion. Considering human tracking as a specific application, we show that hyperpatches can not only be computed automatically during model acquisition from a monocular image sequence, but that they are also extremely appropriate for the task of visual tracking