Context based object categorization: A critical survey

The goal of object categorization is to locate and identify instances of an object category within an image. Recognizing an object in an image is difficult when images include occlusion, poor quality, noise or background clutter, and this task becomes even more challenging when many objects are present in the same scene. Several models for object categorization use appearance and context information from objects to improve recognition accuracy. Appearance information, based on visual cues, can successfully identify object classes up to a certain extent. Context information, based on the interaction among objects in the scene or global scene statistics, can help successfully disambiguate appearance inputs in recognition tasks. In this work we address the problem of incorporating different types of contextual information for robust object categorization in computer vision. We review different ways of using contextual information in the field of object categorization, considering the most common levels of extraction of context and the different levels of contextual interactions. We also examine common machine learning models that integrate context information into object recognition frameworks and discuss scalability, optimizations and possible future approaches.     

M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

When occlusion is minimal, a single camera is generally sufficient to detect and track objects. However, when the density of objects is high, the resulting occlusion and lack of visibility suggests the use of multiple cameras and collaboration between them so that an object is detected using information available from all the cameras in the scene.In this paper, we present a system that is capable of segmenting, detecting and tracking multiple people in a cluttered scene using multiple synchronized surveillance cameras located far from each other. The system is fully automatic, and takes decisions about object detection and tracking using evidence collected from many pairs of cameras. Innovations that help us tackle the problem include a region-based stereo algorithm capable of finding 3D points inside an object knowing only the projections of the object (as a whole) in two views, a segmentation algorithm using bayesian classification and the use of occlusion analysis to combine evidence from different camera pairs.The system has been tested using different densities of people in the scene. This helps us determine the number of cameras required for a particular density of people. Experiments have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.     

Tracking multiple people with recovery from partial and total occlusion

Robust tracking of multiple people in video sequences is a challenging task. In this paper, we present an algorithm for tracking faces of multiple people even in cases of total occlusion. Faces are detected first; then a model for each person is built. The models are handed over to the tracking module which is based on the mean shift algorithm, where each face is represented by the non-parametric distribution of the colors in the face region. The mean shift tracking algorithm is robust to partial occlusion and rotation, and is computationally efficient, but it does not deal with the problem of total occlusion. Our algorithm overcomes this problem by detecting the occlusion using an occlusion grid, and uses a non-parametric distribution of the color of the occluded person's cloth to distinguish that person after the occlusion ends. Our algorithm uses the speed and the trajectory of each occluded person to predict the locations that should be searched after occlusion ends. It integrates multiple features to handle tracking multiple people in cases of partial and total occlusion. Experiments on a large set of video clips demonstrate the robustness of the algorithm, and its capability to correctly track multiple people even when faces are temporarily occluded by other faces or by other objects in the scene.     

Collision Detection between Robot Arms and People

As the result of an increasing number of robots performing tasks in a range of human life activites, human–robot interaction has become a very active research field. Safety of people around robots is a major concern. This paper presents some research in this context: our aim is to avoid mechanical injure of people interacting with robots. We approach the collision detection problem in a scene with people and several moving robot arms. Fast collision detection for practical motion planning depends on an adequate spatial representation for the objects involved in the scene. The authors have previosly proposed a system that automatically generates a hierarchy of approximations for general objects. The spatial model has interesting properties and has been used in efficient collision detection algorithms between moving robots [8]. In spatial representations, there is a trade-off between generality and efficiency. Some existing approaches claim to be general but they are less efficient. In this paper, we present two extensions to the spatial model. First, the system can deal with a general class of objects, those that are composed of nonhomogeneous generalized cylinders. Secondly, a simple method for automatic converting from a polyhedral representation to such a generalized cylinder is presented. Therefore, we enhance the generality of the system but without compromising the efficiency. With these extensions virtually any object can be dealt with, and particularly those composing the human body.     

一种新的基于在线采样的多物体跟踪方法

跟踪多个运动物体,尤其是在遮挡过程中跟踪多个运动物体,是计算机视觉领域一个重要但具有挑战性的问题.该文提出了一种新的在线采样、更新学习和分类的跟踪框架来处理多物体跟踪问题.首先,对遮挡发生前若干帧的各物体进行块采样,作为训练样本进行在线分类器设计.各帧的物体区域也在线进行块采样,并用这些分类器来进行分类标号.如果遮挡没有发生,一些新的训练样本被添加用来更新分类器.当遮挡发生时,根据标号结果,前景区域被分割成多个目标物体.和以往方法相比,新方法不依赖于一些假设条件,如场景深度信息、物体的先验模型(比如形状、种类、区域内颜色各向同性、运动规律等),具有更好的适应能力.实验结果验证了该文方法的稳定性和有效性.     

Learning the Semantic Landscape: embedding scene knowledge in object tracking

The accuracy of object tracking methodologies can be significantly improved by utilizing knowledge about the monitored scene. Such scene knowledge includes the homography between the camera and ground planes and the occlusion landscape identifying the depth map associated with the static occlusions in the scene. Using the ground plane, a simple method of relating the projected height and width of people objects to image location is used to constrain the dimensions of appearance models. Moreover, trajectory modeling can be greatly improved by performing tracking on the ground-plane tracking using global real-world noise models for the observation and dynamic processes. Finally, the occlusion landscape allows the tracker to predict the complete or partial occlusion of object observations. To facilitate plug and play functionality, this scene knowledge must be automatically learnt. The paper demonstrates how, over a sufficient length of time, observations from the monitored scene itself can be used to parameterize the semantic landscape.     

抗遮挡的自适应运动目标跟踪方法

针对复杂场景中的目标遮挡问题,提出一种基于均值漂移(Mean shift)和轨迹校正的自适应目标跟踪方法.由于Mean shift迭代易陷入局部最优点,这里引入Kalman滤波器以预测和校正目标运动轨迹,并根据迭代轨迹误差校正协方差,使得跟踪器在多峰值非高斯分布的复杂环境下也能收敛到全局最优点.基于Bhattacharrya系数计算色彩x、y方向分量相似度,并根据邻帧分量相似度偏差自适应调整相似度融合权值.综合当前帧和前面帧作用更新目标运动状态、特征和尺度模型.实验结果表明提出的方法对于静态场景遮挡和目标间互遮挡、部分和全部遮挡下的目标跟踪均具有鲁棒的跟踪性能.     

Describing Visual Scenes Using Transformed Objects and Parts

We develop hierarchical, probabilistic models for objects, the parts composing them, and the visual scenes surrounding them. Our approach couples topic models originally developed for text analysis with spatial transformations, and thus consistently accounts for geometric constraints. By building integrated scene models, we may discover contextual relationships, and better exploit partially labeled training images. We first consider images of isolated objects, and show that sharing parts among object categories improves detection accuracy when learning from few examples. Turning to multiple object scenes, we propose nonparametric models which use Dirichlet processes to automatically learn the number of parts underlying each object category, and objects composing each scene. The resulting transformed Dirichlet process (TDP) leads to Monte Carlo algorithms which simultaneously segment and recognize objects in street and office scenes.     

Grounding semantic categories in behavioral interactions: Experiments with 100 objects

From an early stage in their development, human infants show a profound drive to explore the objects around them. Research in psychology has shown that this exploration is fundamental for learning the names of objects and object categories. To address this problem in robotics, this paper presents a behavior-grounded approach that enables a robot to recognize the semantic labels of objects using its own behavioral interaction with them. To test this method, our robot interacted with 100 different objects grouped according to 20 different object categories. The robot performed 10 different behaviors on them, while using three sensory modalities (vision, proprioception and audio) to detect any perceptual changes. The results show that the robot was able to use multiple sensorimotor contexts in order to recognize a large number of object categories. Furthermore, the category recognition model presented in this paper was able to identify sensorimotor contexts that can be used to detect specific categories. Most importantly, the robot’s model was able to reduce exploration time by half by dynamically selecting which exploratory behavior should be applied next when classifying a novel object.     

利用部分观测跟踪被遮挡的目标

提出了一个在单个固定摄像机下进行多目标跟踪的方法.利用亮度和色度混合模型和卡尔曼滤波器来检测跟踪目标,为了利于预测和解释被遮挡的物体,建立了场景的模型.在遮挡的情况下,和传统的盲跟踪不同,本文中的目标状态是由可用的部分观测来估计的.对目标的观测取决于预测、前景观测和场景模型.这使得本文算法在定性或定量的分析下都表现出更加鲁棒的性能.     

A Bayesian plan-view map based approach for multiple-person detection and tracking

This work proposes a novel approach for people detection and tracking in colour-with-depth sequences using a particle filtering approach. Detection and tracking are performed in plan-view maps integrating occupancy and height information with a novel plan-view map representation for colour information. Using the three maps, we propose a multiple particle filtering algorithm for people detection and tracking. The observation model proposed integrates information from the three maps so that people with different coloured clothes are not confused even when they interact at close distances. To avoid the coalescence problem when people with similar coloured clothes approach each other, the weight of particles is modified by an interaction factor that combines colour and position information. The algorithm also avoids the coalescence problem in case of total occlusion by means of an occlusion detection and recovering mechanism. Finally, a solution is proposed to improve the exponential complexity of multiple particle filters so that the algorithm proposed has linear complexity.The approach proposed has been tested in several colour-with-depth sequences where people move and interact freely in the environment. In the sequences, people walk at different distances, cross their paths causing frequent occlusions, jump, run and have close interactions such as shaking hands or embracing each other. The experimental results show that our proposal is able to detect and keep track of every person with a low error and deals with partial and total occlusions. Besides, the detection and tracking techniques presented are appropriate for large tracking problems in real-time applications since their complexity is linear, are suitable for parallel processing and allow the integration of information provided by multiple stereo vision sensors.     

Occlusion-aware multi-view reconstruction of articulated objects for manipulation

We present an algorithm called Procrustes-Lo-RANSAC (PLR) to recover complete 3D models of articulated objects. Structure-from-motion techniques are used to capture 3D point cloud models of an object in two different configurations. Procrustes analysis, combined with a locally optimized RANSAC sampling strategy, facilitates a straightforward geometric approach to recovering the joint axes, as well as classifying them automatically as either revolute or prismatic. With the resulting articulated model, a robotic system is then able to manipulate the object along its joint axes at a specified grasp point in order to exercise its degrees of freedom. Because the models capture all sides of the object, they are occlusion-aware, meaning that the robot has knowledge of parts of the object that are not visible in the current view. Our algorithm does not require prior knowledge of the object, nor does it make any assumptions about the planarity of the object or scene. Experiments with a PUMA 500 robotic arm demonstrate the effectiveness of the approach on a variety of real-world objects containing both revolute and prismatic joints.     

Hierarchical framework for robust and fast multiple-target tracking in surveillance scenarios

Multiple-target tracking is a challenging field specially when dealing with uncontrolled scenarios. Two common approaches are often used, one based on low-level techniques to detect each object size, position and velocity, and other based on high-level techniques that deal with object appearance. None of these methods can deal with all possible problems in multiple-target tracking: environment occlusions, both total and partial, and collisions, such as grouping and splitting events. So one solution is to merge these techniques to improve their performance. Based on an existing hierarchical architecture, we present a novel technique that can deal with all the mentioned problems in multiple tracking targets. Blob detection, low-level tracking using adaptive filters, high-level tracking based on a fixed pool of histograms and an event management that can detect every collision event and performs occlusion recovery are used to be able to track every object during the time they appear within the scene. Experimental results show the performance of this technique under multiple situations, being able to track every object in the scene without losing their initial identification. The speed processing is higher than 50 frames, which allows it to be used under real-time scenarios.     

IVIS: An Integrated Volumetric Inspection System

A novel approach to automated visual inspection based on comparing a volumetric model of a reference object to a volumetric model of an actual object which is iteratively created from sensor data is presented. The use of volumetric models gives this approach a number of distinct advantages over more traditional surface-based methods. First, there is no need to identify "features," which is important when inspecting objects whose features are difficult to identify. Second, volumetric inspection lends itself naturally to multisensor applications. Finally, true 3-D comparisons of the reference and sensed objects can be easily carried out using volumetric models.     

Integrating Occlusion Culling and Levels of Detail through Hardly-Visible Sets

Occlusion culling and level-of-detail rendering have become two powerful tools for accelerating the handling of very large models in real-time visualization applications. We present a framework that combines both techniques to improve rendering times. Classical occlusion culling algorithms compute potentially visible sets (PVS), which are supersets of the sets of visible polygons. The novelty of our approach is to estimate the degree of visibility of each object of the PVS using synthesized coarse occluders. This allows to arrange the objects of each PVS into several Hardly-Visible Sets (HVS) with similar occlusion degree. According to image accuracy and frame rate requirements, HVS provide a way to avoid sending to the graphics pipeline those objects whose pixel contribution is low due to partial occlusion. The image error can be bounded by the user at navigation time. On the other hand, as HVS offer a tighter estimation of the pixel contribution for each scene object, it can be used for a more convenient selection of the level-of-detail at which objects are rendered. In this paper, we describe the new framework technique, provide details of its implementation using a visibility octree as the chosen occlusion culling data structure and show some experimental results on the image quality.     

Constraint-based sensor planning for scene modeling

We describe an automated scene modeling system that consists of two components operating in an interleaved fashion: an incremental modeler that builds solid models from range imagery; and a sensor planner that analyzes the resulting model and computes the next sensor position. This planning component is target-driven and computes sensor positions using model information about the imaged surfaces and the unexplored space in a scene. The method is shape-independent and uses a continuous-space representation that preserves the accuracy of sensed data. It is able to completely acquire a scene by repeatedly planning sensor positions, utilizing a partial model to determine volumes of visibility for contiguous areas of unexplored scene. These visibility volumes are combined with sensor placement constraints to compute sets of occlusion-free sensor positions that are guaranteed to improve the quality of the model. We show results for the acquisition of a scene that includes multiple, distinct objects with high occlusion     

Probabilistic people tracking with appearance models and occlusion classification: The AD-HOC system

AD-HOC (Appearance Driven Human tracking with Occlusion Classification) is a complete framework for multiple people tracking in video surveillance applications in presence of large occlusions. The appearance-based approach allows the estimation of the pixel-wise shape of each tracked person even during the occlusion. This peculiarity can be very useful for higher level processes, such as action recognition or event detection. A first step predicts the position of all the objects in the new frame while a MAP framework provides a solution for best placement. A second step associates each candidate foreground pixel to an object according to mutual object position and color similarity. A novel definition of non-visible regions accounts for the parts of the objects that are not detected in the current frame, classifying them as dynamic, scene or apparent occlusions. Results on surveillance videos are reported, using in-house produced videos and the PETS2006 test set.     

监控场景中的运动物体提取技术研究

针对监控场景中因存在遮挡而无法有效地提取出完整的运动序列这一问题,提出了一种将ViBe前景检测算法和改进后的粒子滤波跟踪算法相结合的跟踪提取方法。首先用ViBe来提取出场景中所有运动物体的前景轮廓;其次用粒子滤波来检测和跟踪目标物体;最后通过与目标物体的关联轮廓求交运算以及跟踪区域的反馈调节完成对目标物体运动帧序列的提取。当运动物体发生遮挡时,采用将跟踪区域内所检测到的前景轮廓重新加入到目标物体的关联轮廓中以保证后续可以继续用关联轮廓交集来提取。实验结果表明,该方法能够很好地保证提取的质量,并有效地解决了局部遮挡与全局遮挡情况下运动物体完整运动序列的提取。