Combining object and feature dynamics in probabilistic tracking

Objects can exhibit different dynamics at different spatio-temporal scales, a property that is often exploited by visual tracking algorithms. A local dynamic model is typically used to extract image features that are then used as inputs to a system for tracking the object using a global dynamic model. Approximate local dynamics may be brittle—point trackers drift due to image noise and adaptive background models adapt to foreground objects that become stationary—and constraints from the global model can make them more robust. We propose a probabilistic framework for incorporating knowledge about global dynamics into the local feature extraction processes. A global tracking algorithm can be formulated as a generative model and used to predict feature values thereby influencing the observation process of the feature extractor, which in turn produces feature values that are used in high-level inference. We combine such models utilizing a multichain graphical model framework. We show the utility of our framework for improving feature tracking as well as shape and motion estimates in a batch factorization algorithm. We also propose an approximate filtering algorithm appropriate for online applications and demonstrate its application to tasks in background subtraction, structure from motion and articulated body tracking.     

Fusing force and vision feedback for manipulating deformable objects

This article describes a framework that fuses vision and force feedback for the control of highly deformable objects. Deformable active contours, or snakes, are used to visually observe changes in object shape over time. Finite‐element models of object deformations are used with force feedback to predict expected visually observed deformations. Our approach improves the performance of large, complex deformable contour tracking over traditional computer vision tracking techniques. This same approach of combining deformable active contours with finite‐element material models is modified so that a vision sensor, i.e., a charge‐coupled device (CCD) camera, can be used as a force sensor. By visually tracking changes in contours on the object, material deflections can be transformed into applied stress estimates through finite element modeling. Therefore, internal object stresses due to object manipulation can be visually observed and controlled, thus creating a framework for deformable object manipulation. A pinhole camera model is used to accomplish vision and force sensor feedback assimilation from these two sensing modalities during manipulation. © 2001 John Wiley & Sons, Inc.     

Shape and nonrigid motion estimation through physics-basedsynthesis

A physics-based framework for 3-D shape and nonrigid motion estimation for real-time computer vision systems is presented. The framework features dynamic models that incorporate the mechanical principles of rigid and nonrigid bodies into conventional geometric primitives. Through the efficient numerical simulation of Lagrange equations of motion, the models can synthesize physically correct behaviors in response to applied forces and imposed constraints. Applying continuous Kalman filtering theory, a recursive shape and motion estimator that employs the Lagrange equations as a system model is developed. The system model continually synthesizes nonrigid motion in response to generalized forces that arise from the inconsistency between the incoming observations and the estimated model state. The observation forces also account formally for instantaneous uncertainties and incomplete information. A Riccati procedure updates a covariance matrix that transforms the forces in accordance with the system dynamics and prior observation history. Experiments involving model fitting and tracking of articulated and flexible objects from noisy 3-D data are described     

利用部分观测跟踪被遮挡的目标

提出了一个在单个固定摄像机下进行多目标跟踪的方法.利用亮度和色度混合模型和卡尔曼滤波器来检测跟踪目标,为了利于预测和解释被遮挡的物体,建立了场景的模型.在遮挡的情况下,和传统的盲跟踪不同,本文中的目标状态是由可用的部分观测来估计的.对目标的观测取决于预测、前景观测和场景模型.这使得本文算法在定性或定量的分析下都表现出更加鲁棒的性能.     

A 2D/3D model-based object tracking framework

This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in (IEEE Trans. Pattern Anal. Mach. Intell. 18(2) (1996)) is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object's pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.     

Probabilistic people tracking with appearance models and occlusion classification: The AD-HOC system

AD-HOC (Appearance Driven Human tracking with Occlusion Classification) is a complete framework for multiple people tracking in video surveillance applications in presence of large occlusions. The appearance-based approach allows the estimation of the pixel-wise shape of each tracked person even during the occlusion. This peculiarity can be very useful for higher level processes, such as action recognition or event detection. A first step predicts the position of all the objects in the new frame while a MAP framework provides a solution for best placement. A second step associates each candidate foreground pixel to an object according to mutual object position and color similarity. A novel definition of non-visible regions accounts for the parts of the objects that are not detected in the current frame, classifying them as dynamic, scene or apparent occlusions. Results on surveillance videos are reported, using in-house produced videos and the PETS2006 test set.     

Real-Time Tracking of Single and Multiple Objects from Depth-Colour Imagery Using 3D Signed Distance Functions

We describe a novel probabilistic framework for real-time tracking of multiple objects from combined depth-colour imagery. Object shape is represented implicitly using 3D signed distance functions. Probabilistic generative models based on these functions are developed to account for the observed RGB-D imagery, and tracking is posed as a maximum a posteriori problem. We present first a method suited to tracking a single rigid 3D object, and then generalise this to multiple objects by combining distance functions into a shape union in the frame of the camera. This second model accounts for similarity and proximity between objects, and leads to robust real-time tracking without recourse to bolt-on or ad-hoc collision detection.     

Trackability as a cue for potential obstacle identification and 3-D description

In many man-made environments, obstacles in the path of a mobile robot can be characterized asshallow, that is, they have relatively small extent in depth compared to the distance from the camera. We present a framework for segmenting shallow structures from their background over a sequence of images. Shallowness is first quantified asaffine describability. This is embedded in a tracking system within which hypothesized model structures undergo a cycle of prediction and model-matching. Structures emerge either as shallow or nonshallow based on theiraffine trackability. Two major contributions of this work are (i) aggregate object tracking based on 3-D motion and structure constraints in constrast with traditional primitive feature tracking based on image motion heuristics, and (ii) use of temporal behavior for object segmentation and 3-D reconstruction.     

基于视频的三维人体运动跟踪系统的设计与实现

在优化粒子滤波跟踪框架下,设计并实现了一个结合多种图像特征、在多摄像机环境下跟踪人体运动的三维人体运动跟踪系统.通过定义三维人体模型、摄像机模型以及观测似然模型,得到跟踪所需目标函数,并使用优化粒子滤波算法进行求解.实验结果表明,该系统能够对人体运动进行准确的跟踪和三维重建,可应用于体育运动分析和动画制作等领域.     

Object joint detection and tracking using adaptive multiple motion models

This paper deals with the problem of detecting objects that may switch between different motion models. In order to accurately detect these moving objects taking into account possible changing motion models, we propose an adaptive multi-motion model in the joint detection and tracking (JDT) framework. The proposed technique differs from the existing JDT-based methods mainly in two ways. First we express the solution in the JDT framework via a formulation in the multiple motion model setting. Second, we introduce a new motion model prediction function which exploits the correlation between the motion model and object kinematic state. Experiments on both synthetic and real videos demonstrate that the JDT method employing the proposed adaptive multi-motion model can detect objects more accurately than the existing peer methods when objects change their motion models.     

I-VITAL: Information aided visual tracking with adversarial learning

With the advent of convolutional neural networks (CNN), MDNet and the Siamese trackers posed tracking as supervised learning. They model an object’s presence using classification (foreground and background) and location using regression. For the first time, we have brought probability distribution into the CNN framework for tracking. We have selected “Information maximization Generative Adversarial Network (InfoGAN)” to couple the target and background classes with two unique Gaussian distributions. This paper highlights the use of InfoGAN in information extraction & feedback to improve the tracking framework. Specifically, the novel features proposed in this tracking framework are (i) Coupling of unique probability distributions to target and background classes and (ii) Unsupervised tracker status (success/ failure) identification and correction through information feedback. We demonstrated the efficacy of the proposed I-VITAL tracker in visual tracking with experimental comparisons on well-known data sets such as GOT10K, VOT2020, and OTB-2015. Compared with base works, the proposed tracker has improved performance in locating the object of interest.     

基于多任务最小软阈值回归方法的目标跟踪*

在视频跟踪中,模型表示是直接影响跟踪效率的核心问题之一.在随时间和空间变化的复杂数据中学习目标外观模型表示所需的有效模板,从而适应内在或外在因素所引起的目标状态变化是非常重要的.文中详细描述较为鲁棒的目标外观模型表示策略,并提出一种新的多任务最小软阈值回归跟踪算法(MLST).该算法框架将候选目标的观测模型假设为多任务线性回归问题,利用目标模板和独立同分布的高斯-拉普拉斯重构误差线性表示候选目标不同状态下的外观模型,从而跟踪器能够很好地适应各种复杂场景并准确预测每一时刻的真实目标状态.大量实验证明,文中在线学习策略能够充分挖掘目标在不同时刻的特殊状态信息以提高模型表示精度,使得跟踪器保持最佳的状态,从而在一定程度上提高跟踪性能.实验结果显示,本文算法体现较好的鲁棒性并优于一些目前较先进的跟踪算法.     

A variational method for contour tracking via covariance matching

This paper presents a novel formulation for contour tracking.We model the second-order statistics of image regions and perform covariance matching under the variational level set framework.Specifically,covariance matrix is adopted as a visual object representation for partial differential equation(PDE) based contour tracking.Log-Euclidean calculus is used as a covariance distance metric instead of Euclidean distance which is unsuitable for measuring the similarities between covariance matrices,because the matrices typically lie on a non-Euclidean manifold.A novel image energy functional is formulated by minimizing the distance metric between the candidate object region and a given template,and maximizing the one between the background region and the template.The corresponding gradient flow is then derived according to a variational approach,enabling partial differential equations(PDEs) based contour tracking.Experiments on several challenging sequences prove the validity of the proposed method.     

Towards Collaborative Robotics in Top View Surveillance: A Framework for Multiple Object Tracking by Detection Using Deep Learning

Collaborative Robotics is one of the high-interest research topics in the area of academia and industry. It has been progressively utilized in numerous applications, particularly in intelligent surveillance systems. It allows the deployment of smart cameras or optical sensors with computer vision techniques, which may serve in several object detection and tracking tasks. These tasks have been considered challenging and high-level perceptual problems, frequently dominated by relative information about the environment, where main concerns such as occlusion, illumination, background, object deformation, and object class variations are commonplace. In order to show the importance of top view surveillance, a collaborative robotics framework has been presented. It can assist in the detection and tracking of multiple objects in top view surveillance. The framework consists of a smart robotic camera embedded with the visual processing unit. The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization. The detection models are further combined with different tracking algorithms, including GOTURN, MEDIANFLOW, TLD, KCF, MIL, and BOOSTING. These algorithms, along with detection models, help to track and predict the trajectories of detected objects. The pre-trained models are employed; therefore, the generalization performance is also investigated through testing the models on various sequences of top view data set. The detection models achieved maximum True Detection Rate 93% to 90% with a maximum 0.6% False Detection Rate. The tracking results of different algorithms are nearly identical, with tracking accuracy ranging from 90% to 94%. Furthermore, a discussion has been carried out on output results along with future guidelines.      

有环境约束时完成一族跟踪任务的力控制

本文为工业机器人提出了一种极点配置控制法，这种控制方法的优点有：一是它的积分作用消除了机器人的微小扰动和稳态误差；二是能任意设置系统的极点，因此能保证闭环系统的稳定性和规定状态变量的暂态响应；三是加入了加速度反馈，抑制了由电枢电感所引起的机械手的振动，最后，给出了ＰＵＭＡ５６２机器人的计算机仿真和实验结果验证了此控制法的有效性。     

On the robust control of two manipulators holding a rigid object

In this paper, a control architecture is developed for the closed chain motion of two six-joint manipulators holding a rigid object in a three-dimensional workspace. Dynamic and kinematic constraints are combined with the equations of motion of the manipulators to obtain a dynamical model of the entire system in the joint space. Reduced-order dynamic equations are then developed with regard to the position and force control variables. Robust control laws are then determined such that the force and position control design is decoupled. The control laws that will be discussed are: a robust position tracking controller that yields an exponentially stable position tracking error result, and a robust force tracking controller that yields adjustable bounds on the force tracking error.     

Adaptive multi-cue tracking by online appearance learning

This paper proposes a multi-cue based appearance learning algorithm for object tracking. In each frame, the target object is represented by different cues in the image-as-matrix form. This representation can describe the target from different perspectives and can preserve the spatial correlation information inside the target region. Based on these cues, multiple appearance models are learned online by bilinear subspace analysis to account for the target appearance variations over time. Tracking is formulated within the Bayesian inference framework, in which the observation model is constructed by fusing all the learned appearance models. The combination of online appearance modeling and weight update of each appearance model can adapt our tracking algorithm to both the target and background changes. We test our algorithm on a variety of challenging sequences by tracking car, face, pedestrian, and so on. Experimental results and comparisons to several state-of-the-art methods show improved tracking performance.     

Visual tracking via dynamic tensor analysis with mean update

The appearance model is an important issue in the visual tracking community. Most subspace-based appearance models focus on the time correlation between the image observations of the object, but the spatial layout information of the object is ignored. This paper proposes a robust appearance model for visual tracking which effectively combines the spatial and temporal eigen-spaces of the object in a tensor reconstruction way. In order to capture the variations in object appearance, an incremental updating strategy is developed to both update the eigen-space and mean of the object. Experimental results demonstrate that, compared with the state-of-the-art appearance models in the tracking literature, the proposed appearance model is more robust and effective.     

全局搜索和多实例判别特征的长时跟踪方法

长时目标跟踪相对于短时目标跟踪仍然是一个巨大的挑战. 然而现有的长时跟踪算法通常在面对目标频繁出现消失、目标外观发生剧变等挑战中表现不佳. 本文提出了一种基于局部搜索模块和全局搜索跟踪模块的全新、鲁棒且实时的长时跟踪框架. 局部搜索模块利用TransT短时跟踪器生成一系列候选框, 并通过置信度评分确定最佳候选框. 针对全局重新检测开发了一个新颖的全局搜索跟踪模块, 以Faster R-CNN为基础模型, 在RPN阶段与R-CNN阶段引入非局部操作和多级实例特征融合模块, 以充分挖掘目标实例级特征. 为了改进全局搜索跟踪模块的性能, 设计了双模板更新策略来提升跟踪器的鲁棒能力. 通过使用不同时间点上更新的模板能够更好地适应目标的变化. 根据局部或全局置信度分数判断目标是否存在, 并在下一帧中选择局部或全局搜索跟踪策略. 同时能够为局部搜索模块估计目标的位置和大小. 此外还为全局搜索跟踪器引入了排名损失函数, 隐式学习了区域提议与原始查询目标的相似度. 通过在多个跟踪数据集上进行大量实验对提出的跟踪框架进行了广泛评估. 结果一致表明, 本文提出的跟踪框架实现了令人满意的性能.