An appearance-based approach for consistent labeling of humans and objects in video

In this paper, we present an approach for consistently labeling people and for detecting human–object interactions using mono-camera surveillance video. The approach is based on a robust appearance-based correlogram model combined with histogram information to model color distributions of people and objects in the scene. The models are dynamically built from non-stationary objects, which are the outputs of background subtraction, and are used to identify objects on a frame-by-frame basis. We are able to detect when people merge into groups and to segment them even during partial occlusion. We can also detect when a person deposits or removes an object. The models persist when a person or object leaves the scene and are used to identify them when they reappear. Experiments show that the models are able to accommodate perspective foreshortening that occurs with overhead camera angles, as well as partial occlusion. The results show that this is an effective approach that is able to provide important information to algorithms performing higher-level analysis, such as activity recognition, where human–object interactions play an important role.     

W4: real-time surveillance of people and theiractivities

W⁴ is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. W⁴ employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. W⁴ can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. W⁴ can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320×240 resolution images on a 400 MHz dual-Pentium II PC     

M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene

When occlusion is minimal, a single camera is generally sufficient to detect and track objects. However, when the density of objects is high, the resulting occlusion and lack of visibility suggests the use of multiple cameras and collaboration between them so that an object is detected using information available from all the cameras in the scene.In this paper, we present a system that is capable of segmenting, detecting and tracking multiple people in a cluttered scene using multiple synchronized surveillance cameras located far from each other. The system is fully automatic, and takes decisions about object detection and tracking using evidence collected from many pairs of cameras. Innovations that help us tackle the problem include a region-based stereo algorithm capable of finding 3D points inside an object knowing only the projections of the object (as a whole) in two views, a segmentation algorithm using bayesian classification and the use of occlusion analysis to combine evidence from different camera pairs.The system has been tested using different densities of people in the scene. This helps us determine the number of cameras required for a particular density of people. Experiments have also been conducted to verify and quantify the efficacy of the occlusion analysis scheme.     

The MIDAS touch: Thermal dissipation resulting from everyday interactions as a sensing modality

We contribute MIDAS as a novel sensing solution for characterizing everyday objects using thermal dissipation. MIDAS takes advantage of the fact that anytime a person touches an object it results in heat transfer. By capturing and modeling the dissipation of the transferred heat, e.g., through the decrease in the captured thermal radiation, MIDAS can characterize the object and determine its material. We validate MIDAS through extensive empirical benchmarks and demonstrate that MIDAS offers an innovative sensing modality that can recognize a wide range of materials – with up to 83% accuracy – and generalize to variations in the people interacting with objects. We also demonstrate that MIDAS can detect thermal dissipation through objects, up to 2 mm thickness, and support analysis of multiple objects that are interacted with.     

Robust real-time periodic motion detection, analysis, andapplications

We describe new techniques to detect and analyze periodic motion as seen from both a static and a moving camera. By tracking objects of interest, we compute an object's self-similarity as it evolves in time. For periodic motion, the self-similarity measure is also periodic and we apply time-frequency analysis to detect and characterize the periodic motion. The periodicity is also analyzed robustly using the 2D lattice structures inherent in similarity matrices. A real-time system has been implemented to track and classify objects using periodicity. Examples of object classification (people, running dogs, vehicles), person counting, and nonstationary periodicity are provided     

Probabilistic Methods for Finding People

Finding people in pictures presents a particularly difficult object recognition problem. We show how to find people by finding candidate body segments, and then constructing assemblies of segments that are consistent with the constraints on the appearance of a person that result from kinematic properties. Since a reasonable model of a person requires at least nine segments, it is not possible to inspect every group, due to the huge combinatorial complexity.We propose two approaches to this problem. In one, the search can be pruned by using projected versions of a classifier that accepts groups corresponding to people. We describe an efficient projection algorithm for one popular classifier, and demonstrate that our approach can be used to determine whether images of real scenes contain people.The second approach employs a probabilistic framework, so that we can draw samples of assemblies, with probabilities proportional to their likelihood, which allows to draw human-like assemblies more often than the non-person ones. The main performance problem is in segmentation of images, but the overall results of both approaches on real images of people are encouraging.     

A novel cooperative caching algorithm for massive P2P caches

For an ISP (Internet Service Provider) that has deployed P2P caches in more than one ASs (autonomous systems), cooperative caching which makes their caches cooperate with each other can save more cost of carrying P2P traffic than independent caching. However, existing cooperative caching algorithms only use objects’ popularity as the measurement to decide which objects should be cached, and cost on intra-ISP links that has great impact on the benefits of cooperative caching is not considered. In this paper, we first model the cooperative caching problem as a NP-Complete problem, which is based on our analysis about the cost of serving requests with consideration of both the objects’ popularity and the cost on intra-ISP links. Then we propose a novel cooperative caching algorithm named cLGV (Cooperative, Lowest Global Value). The cLGV algorithm uses a new concept global value to estimate the benefits of caching or replacing an object in the cooperative caching system, and the global value of each object is evaluated according to not only objects’ popularity in each AS but also cost on intra-ISP links among ASs. Results of both synthetic and real traces driven simulations indicate that our cLGV algorithm can save the cost of carrying P2P traffic at least 23 % higher than that of existing cooperative caching algorithms.     

一种室内动态场景下RGB-D SLAM的前端优化方法

针对动态物体会导致位姿估计误差过大的问题,提出一种适用于动态场景下的RGB-D SLAM算法。首先,将目标检测的结果根据物体是否具有自主移动能力划分为动态物体与潜在动态物体,并使用大津法在对应的深度图中完成检测框内区域的前、背景分割;然后,结合检测框的位置坐标与深度信息确定潜在动态物体与动态物体在三维空间内是否关联,初步筛选出具有一定运动概率的潜在动态物体;最后,通过运动一致性检测算法来最终确定是否将潜在动态物体作为动态物体处理。实验部分选用公开的TUM数据集,结果表明,该算法在保持实时性的同时具有较高的定位精度和鲁棒性。     

Tracking multiple people with recovery from partial and total occlusion

Robust tracking of multiple people in video sequences is a challenging task. In this paper, we present an algorithm for tracking faces of multiple people even in cases of total occlusion. Faces are detected first; then a model for each person is built. The models are handed over to the tracking module which is based on the mean shift algorithm, where each face is represented by the non-parametric distribution of the colors in the face region. The mean shift tracking algorithm is robust to partial occlusion and rotation, and is computationally efficient, but it does not deal with the problem of total occlusion. Our algorithm overcomes this problem by detecting the occlusion using an occlusion grid, and uses a non-parametric distribution of the color of the occluded person's cloth to distinguish that person after the occlusion ends. Our algorithm uses the speed and the trajectory of each occluded person to predict the locations that should be searched after occlusion ends. It integrates multiple features to handle tracking multiple people in cases of partial and total occlusion. Experiments on a large set of video clips demonstrate the robustness of the algorithm, and its capability to correctly track multiple people even when faces are temporarily occluded by other faces or by other objects in the scene.     

Inferring additional knowledge from QTCN relations

It is widely held that people tend to use qualitative rather than quantitative phrases when raising or answering questions about moving objects. Queries about whether an object is moving towards or away from another object or whether objects are getting closer to each other or further away from each other, require qualitative responses. This characteristic should be reflected in a calculus to be used to describe and reason about continuously moving objects. In this paper, we present a qualitative trajectory calculus of relations between two disjoint moving objects, whose movement is constrained by a network. The proposed calculus (QTC_N) is formally introduced and illustrated. Particular attention is placed on how to infer additional knowledge from QTC_N relations by means of composition tables and the transformation of QTC_N relations into relations defined by the Relative Trajectory Calculus on Networks (RTC_N).     

Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments

Independent travel is a well-known challenge for blind and visually impaired persons. In this paper, we propose a proof-of-concept computer vision-based wayfinding aid for blind people to independently access unfamiliar indoor environments. In order to find different rooms (e.g. an office, a laboratory, or a bathroom) and other building amenities (e.g. an exit or an elevator), we incorporate object detection with text recognition. First, we develop a robust and efficient algorithm to detect doors, elevators, and cabinets based on their general geometric shape, by combining edges and corners. The algorithm is general enough to handle large intra-class variations of objects with different appearances among different indoor environments, as well as small inter-class differences between different objects such as doors and door-like cabinets. Next, to distinguish intra-class objects (e.g. an office door from a bathroom door), we extract and recognize text information associated with the detected objects. For text recognition, we first extract text regions from signs with multiple colors and possibly complex backgrounds, and then apply character localization and topological analysis to filter out background interference. The extracted text is recognized using off-the-shelf optical character recognition software products. The object type, orientation, location, and text information are presented to the blind traveler as speech.     

Specific object retrieval based on salient regions

In this paper, we present an image retrieval technique for specific objects based on salient regions. The salient regions we select are invariant to geometric and photometric variations. Those salient regions are detected based on low level features, and need to be classified into different types before they can be applied on further vision tasks. We first classify the selected regions into four types including blobs, edges and lines, textures, and texture boundaries, by using the correlations with the neigbouring regions. Then, some specific region types are chosen for further object retrieval applications. We observe that regions selected from images of the same object are more similar to each other than regions selected from images of different objects. Correlation is used as the similarity measure between regions selected from different images. Two images are considered to contain the same object, if some regions selected from the first image are highly correlated to some regions selected from the second image. Two data sets are employed for experiment: the first data set contains human face images of a number of different people and is used for testing the retrieval algorithm on distinguishing specific objects of the same category; and the second data set contains images of different objects and is used for testing the retrieval algorithm on distinguishing objects of different categories. The results show that our method is very effective on specific object retrieval.     

基于混合子空间去噪的免携带设备目标定位

免携带设备目标定位是利用目标人物对无线通信链路产生的阴影衰落来确定目标的位置,然而当环境发生变化时也会引起无线链路信号强度的变化,会影响定位精度。针对传统子空间去噪方法的不足,本文与指纹法相结合探讨了基于子空间分解的小波去噪方法。通过研究静态环境下的噪声特征,选取最大特征值作为信号分量的阈值,自适应地提取目标信号,有效地消除环境变化产生的影响。在线阶段通过计算实时接收信号强度和射频地图中数据信息之间的核距离进行匹配估计出目标的位置。最后通过实验对本文算法进行了仿真,结果表明本文提出的算法相比传统已有算法能达到更好的定位精度。     

运动目标检测算法在军用视频监控系统中的应

在充分分析了现有算法在复杂背景下所存在的不足的基础上,提出了一种适用于军用视频监控系统的基于视频序列象素时空相关性检测的运动目标检测方法.该方法首先用每一帧中目标的空间相关性检测出目标;再用序列图像中目标的时间相关性检验目标的真实性,从而最终确定是否有运动目标.实际应用表明,采用该算法的系统能很好地检测出运动目标,并具有较强的抗干扰能力.     

监控系统中的多摄像头协同算法

给出了一个分布式多摄像头监控系统结构,描述了一种使用路径模型和目标外形变化估计的数据融合方案,提出了一种新的在分布式监控系统中的多摄像头协同算法(PDA,PriorityandDistanceAlgorithm)。提出的算法基于多摄像头数据融合结果,根据任务优先级、目标与摄像头之间的距离及目标的可见性情况,分配摄像头给目标,其特点是使系统中有高优先级并距离摄像头最近的可见目标优先分配摄像头。实验结果表明提出算法能协同多摄像头可靠地跟踪人。     

监控系统中的多摄像机协同

描述了一个用于室内场合对多个目标进行跟踪的分布式监控系统.该系统由多个廉价的固定镜头的摄像机构成,具有多个摄像机处理模块和一个中央模块用于协调摄像机间的跟踪任务.由于每个运动目标有可能被多个摄像机同时跟踪,因此如何选择最合适的摄像机对某一目标跟踪,特别是在系统资源紧张时,成为一个问题.提出的新算法能根据目标与摄像机之间的距离并考虑到遮挡的情况,把目标分配给相应的摄像机,因此在遮挡出现时,系统能把遮挡的目标分配给能看见目标并距离最近的那个摄像机.实验表明该系统能协调好多个摄像机进行目标跟踪,并处理好遮挡问题.     

Contour tracking with automatic motion model switching

In this paper we present an efficient contour-tracking algorithm which can track 2D silhouette of objects in extended image sequences. We demonstrate the ability of the tracker by tracking highly deformable contours (such as walking people) captured by a static camera. We represent contours (silhouette) of moving objects by using a cubic B-spline. The tracking algorithm is based on tracking a lower dimensional shape space (as opposed to tracking in spline space). Tracking the lower dimensional space has proved to be fast and efficient. The tracker is also coupled with an automatic motion-model switching algorithm, which makes the tracker robust and reliable when the object of interest is moving with multiple motion. The model-based tracking technique provided is capable of tracking rigid and non-rigid object contours with good tracking accuracy.     

Efficient algorithms for common transversals

Suppose we are given a set S of n (possibly intersecting) simple objects in the plane such that, for every pair of objects in S, the intersection of the boundaries of these two objects has O(1) connected components. We consider the problem of determining whether there exists a straight line that goes through every object in S. We give an O(n log n γ (n)) time algorithm for this problem, where γ(n) is a very slowly growing function of n. In many cases, our algorithm runs in O(n log n) time. Previously, only special cases of this problem were considered: the case when every object is a straight-line segment (Edelsbrunner et al., 1982), the case when the objects are equal-radius circles (Bajaj and Li, 1983), and the case when objects all maintain the same orientation (Edelsbrunner, 1985). All these cases follow from our general approach, which places no constraints on the size and/or configuration of the objects in S.