MobileNet-JDE: a lightweight multi-object tracking model for embedded systems

Multimedia Tools and Applications - Multi-object tracking (MOT) is one of the most challenging tasks in the field of computer vision. Although many MOT methods have been proposed in the literature,...     

Transformer-based two-source motion model for multi-object tracking

Recently, benefit from the development of detection models, the multi-object tracking method based on tracking-by-detection has greatly improved performance. However, most methods still utilize traditional motion models for position prediction, such as the constant velocity model and Kalman filter. Only a few methods adopt deep network-based methods for prediction. Still, these methods only exploit the simplest RNN(Recurrent Neural Network) to predict the position, and the position offset caused by the camera movement is not considered. Therefore, inspired by the outstanding performance of Transformer in temporal tasks, this paper proposes a Transformer-based motion model for multi-object tracking. By taking the historical position difference of the target and the offset vector between consecutive frames as input, the model considers the motion of the target itself and the camera at the same time, which improves the prediction accuracy of the motion model used in the multi-target tracking method, thereby improving tracking performance. Through comparative experiments and tracking results on MOTchallenge benchmarks, the effectiveness of the proposed method is proved.

     

Embedded omni-vision navigator based on multi-object tracking

This paper presents an embedded omni-vision navigation system which involves landmark recognition, multi-object tracking, and vehicle localization. A new tracking algorithm, the feature matching embedded particle filter, is proposed. Landmark recognition is used to provide the front-end targets. A global localization method for omni-vision based on coordinate transformation is also proposed. The digital signal processor (DSP) provides a hardware platform for on-board tracker. Dynamic navigator employs DSP tracker to follow the landmarks in real time during the arbitrary movement of the vehicle and computes the position for localization based on time sequence images analysis. Experimental results demonstrated that the navigator can efficiently offer the vehicle guidance.     

Online multi-object tracking using multi-function integration and tracking simulation training

Applied Intelligence - Recently, with the development of deep-learning, the performance of multi-object tracking algorithms based on deep neural networks has been greatly improved. However, most...     

Distributed wide-area multi-object tracking with non-overlapping camera views

We present a distributed system for wide-area multi-object tracking across disjoint camera views. Every camera in the system performs multi-object tracking, and keeps its own trackers and trajectories. The data from multiple features are exchanged between adjacent cameras for object matching. We employ a probabilistic Petri Net-based approach to account for the uncertainties of the vision algorithms (such as unreliable background subtraction, and tracking failure) and to incorporate the available domain knowledge. We combine appearance features of objects as well as the travel-time evidence for target matching and consistent labeling across disjoint camera views. 3D color histogram, histogram of oriented gradients, local binary patterns, object size and aspect ratio are used as the appearance features. The distribution of the travel time is modeled by a Gaussian mixture model. Multiple features are combined by the weights, which are assigned based on the reliability of the features. By incorporating the domain knowledge about the camera configurations and the information about the received packets from other cameras, certain transitions are fired in the probabilistic Petri net. The system is trained to learn different parameters of the matching process, and updated online. We first present wide-area tracking of vehicles, where we used three non-overlapping cameras. The first and the third cameras are approximately 150 m apart from each other with two intersections in the blind region. We also present an example of applying our method to a people-tracking scenario. The results show the success of the proposed method. A comparison between our work and related work is also presented.     

Online multi-object tracking: multiple instance based target appearance model

Multimedia Tools and Applications - The online target specific feature based state estimation method has proved its applicability in video-based multiple objects tracking. This paper proposes a...     

分块多特征自适应融合的多目标视觉跟踪

为提高多目标视觉跟踪算法的实时性和稳定性,提出了分块多特征融合的目标跟踪算法.该算法融合底层颜色、纹理和边缘特征信息,以降低单一目标特征算法容易受复杂环境和目标形变的影响.建立分块目标多特征融合直方图模型,引入目标和背景区分度抑制背景分量,并且结合Kalman滤波器进行预测,在发生遮挡时根据置信度最大子块位置获取遮挡目标位置,实现目标稳定可靠的跟踪.实验结果表明:该算法对每帧图像的平均处理时间为36.2 ms,达到实时性的目的,且算法鲁棒性较强.     

固定场景下多目标运动检测与跟踪

提出了一种多目标运动检测与跟踪方法,先提出一种从有运动目标存在的视频序列中鲁棒获取背景帧的方法,随后根据当前帧和背景帧的亮度偏差及色度偏差值将图像二值化,并进行预处理,然后提出结合全局背景周期更新和局部背景更新的方法来提高检测效果.再通过卡尔曼滤波对多目标进行跟踪,利用颜色直方图和形状模型进行识别.实验结果表明,该方法具有较好的处理效果,满足实时系统的要求.     

Deep learning in multi-object detection and tracking: state of the art

Applied Intelligence - Object detection and tracking is one of the most important and challenging branches in computer vision, and have been widely applied in various fields, such as health-care...     

Camera handoff with adaptive resource management for multi-camera multi-object tracking

Camera handoff is a crucial step to obtain a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are identified across different cameras. However, there exist many unsolved questions in developing an efficient camera handoff algorithm. In this paper, we first design a trackability measure to quantitatively evaluate the effectiveness of object tracking so that camera handoff can be triggered timely and the camera to which the object of interest is transferred can be selected optimally. Three components are considered: resolution, distance to the edge of the camera’s field of view (FOV), and occlusion. In addition, most existing real-time object tracking systems see a decrease in the frame rate as the number of tracked objects increases. To address this issue, our handoff algorithm employs an adaptive resource management mechanism to dynamically allocate cameras’ resources to multiple objects with different priorities so that the required minimum frame rate is maintained. Experimental results illustrate that the proposed camera handoff algorithm can achieve a substantially improved overall tracking rate by 20% in comparison with the algorithm presented by Khan and Shah.     

Coupled multi-object tracking and labeling for vehicle trajectory estimation and matching

Efficient detection and tracking of moving objects in real life conditions is a very challenging research issue, mainly due to occlusions, illumination variations, appearance (disappearance) of new (existing) objects and overlapping issues. In this paper, we address these difficulties by incorporating non-linear and recursive identification mechanisms in motion-based detection and tracking algorithms. Non-linearity allows correct identification of object of complex visual properties while the adaptability makes the proposed scheme able to update its behaviour to the dynamic environmental changes. In addition, in this paper, we introduce the concept of polar spectrum which is a measure for determining the deviation of a vehicle trajectory from an ideal trace. The proposed methods (object tracking and trajectory matching) are applied in survey engineering problems dealing with safe design road turns. In particular, the automatically detected trajectory of a moving vehicle is compared with the ideal trace, through the polar spectrum measure, to determine the safety of a road turn. This trace is also compared with the one manually derived using photogrammetric algorithms and a small error is obtained verifying the efficiency of the method.     

Grid-based multi-object tracking with Siamese CNN based appearance edge and access region mechanism

Receiving growing attention for its various applications during the last few years, multi-object tracking remains a complex and challenging problem. Conventional grid-based tracking method is an efficient and effective method to tackle multi-object tracking, whose performance can be further boosted by intuitively taking into account the appearance similarity information yet. Therefore, we introduce appearance similarity edge into the grid-based method, where a Siamese network is utilized to produce the proposed similarity edge. In addition, we build a grid model with hexagonal cells and propose an access region mechanism including accessible area definition and an automatic-generation approach for entrance/exit grids. Since our tracking framework follows ’tracking-by-detection’ paradigm, the corresponding detection information is available to be integrated into access region mechanism, which will facilitate appropriate grid modeling. We verify the proposed Siamese network based appearance edge and access region mechanism through the experiments on some popular datasets like PETS-09, KITTI.

     

Collaborative video searching on a tabletop

Almost all system and application design for multimedia systems is based around a single user working in isolation to perform some task yet much of the work for which we use computers to help us, is based on working collaboratively with colleagues. Groupware systems do support user collaboration but typically this is supported through software and users still physically work independently. Tabletop systems, such as the DiamondTouch from MERL, are interface devices which support direct user collaboration on a tabletop. When a tabletop is used as the interface for a multimedia system, such as a video search system, then this kind of direct collaboration raises many questions for system design. In this paper we present a tabletop system for supporting a pair of users in a video search task and we evaluate the system not only in terms of search performance but also in terms of user–user interaction and how different user personalities within each pair of searchers impacts search performance and user interaction. Incorporating the user into the system evaluation as we have done here reveals several interesting results and has important ramifications for the design of a multimedia search system.     

Cooperative processing based on posture change detection and trajectory estimation for unknown multi-object tracking

ABSTRACT

Tracking of moving objects is a very important step in building an intelligent video surveillance system. The movement of non-rigid objects, appearance variations and luminosity changes make tracking even more difficult. This paper proposes a new automatic multi-target tracking system that can deal with the most confronted problems without any prior knowledge of the characteristics of objects. The system is a combination between classification, learning and tracking in a parallel architecture that allows the three tasks to be performed separately and efficiently to make the most of this combination. The permanent learning of the classifier guarantees the efficiency of the latter compared to the posture changes of moving objects. The classifier sends the new posture changes with a high degree of confidence as a new learning data. This cyclic aspect forces the system to adapt to all possible posture changes. In the case of occlusion, the system uses the estimated information of the trajectories to correct or cancel the learning process. The filtering process prevents the classifier from falling into a false classification, which significantly increases the system adaptability to the environment. Tests carried out on the CAVIAR and MOT16 datasets showed the efficiency and effectiveness of the proposed system.     

Co-located collaborative visual analytics around a tabletop display

Co-located collaboration can be extremely valuable during complex visual analytics tasks. We present an exploratory study of a system designed to support collaborative visual analysis tasks on a digital tabletop display. Fifteen participant pairs employed Cambiera, a visual analytics system, to solve a problem involving 240 digital documents. Our analysis, supported by observations, system logs, questionnaires, and interview data, explores how pairs approached the problem around the table. We contribute a unique, rich understanding of how users worked together around the table and identify eight types of collaboration styles that can be used to identify how closely people work together while problem solving. We show how the closeness of teams’ collaboration and communication influenced how they performed on the task overall. We further discuss the role of the tabletop for visual analytics tasks and derive design implications for future co-located collaborative tabletop problem solving systems.     

Lumisight Table: an interactive view-dependent tabletop display

A novel tabletop display provides different images to different users surrounding the system. It can also capture users' gestures and physical objects on the tabletop. The Lumisight Table approach is based on the optical design of a special screen system composed of a building material called Lumisty and a Fresnel lens. The system combines these films and a lens with four projectors to display four different images, one for each user's view. In addition, we need appropriate input methods for this display media. In the current state of the project, we can control computers by placing physical objects on the display or placing our hands over the display. This screen system also makes it possible to use a camera to capture the appearance of the tabletop from inside of the system. Our other main idea is to develop attractive and specialized applications on the Lumisight Table, including games and applications for computer-supported cooperative-work (CSCW) environments. The projected images can be completely different from each other, or partially identical and partially different. Users can share the identical parts as public information, because all users can see it. This article is available with a short video documentary on CD-ROM.     

Storage bins: mobile storage for collaborative tabletop displays

The ability to store resource items anywhere in the workspace and move them around can be critical for coordinating task and group interactions on a table. However, existing casual storage techniques for digital workspaces only provide access to stored items at the periphery of the workspace, potentially compromising collaborative interactions at a digital tabletop display. To facilitate this storage behavior in a digital tabletop workspace, we developed the storage bin mobile storage mechanism, which combines the space-preserving features of existing peripheral storage mechanisms with the capability to relocate stored items in the workspace. A user study explores the utility of storage bins on tabletop display collaboration.     

HoloTabletop: an anamorphic illusion interactive holographic-like tabletop system

HoloTabletop is a low-cost holographic-like tabletop interactive system. This system analyzes user’s head position and gaze location in a real time setting and computes the corresponding anamorphic illusion image. The anamorphic illusion image is displayed on a 2D horizontally-located monitor, yet offers stereo vision to the user. The user is able to view and interact with the 3D virtual objects without wearing any special glasses or devices. The experimental results and user studies verify that the proposed HoloTabletop system offers excellent stereo vision while no visual fatigue will be caused to human eyes. This system is a great solution for many interactive applications such as 3D board games and stereo map browsing.