面向服务机器人的多行人检测与跟踪

服务机器人在给人提供帮助,带来生活便利的同时,需要检测并跟踪行人.然而,环境复杂,多个行人之间存在遮挡等问题,给行人的检测与跟踪带来了挑战.在行人检测方面,本文提出了最近邻方法融合激光人腿检测和Kinect人体检测的结果,有效改善了行人检测的精度和完整性.针对多行人跟踪,本文提出了一种改进的粒子滤波算法对行人的位置和速度进行了估计,克服了传统粒子滤波算法计算量大,重采样阶段粒子贫化的缺点.最后,在实际场景中采用改造的turtlebot机器人进行了测试,并进行了计算机可视化,实验结果证明本文提出的方法具有很好的准确性,实时性和鲁棒性.     

Tracking people in three dimensions using a hierarchical model of dynamics

We propose a novel hierarchical model of human dynamics for view independent tracking of a human figure in monocular video sequences. The model is trained using real data from a collection of people. The top of the hierarchy contains information about the whole body. The lower levels of the hierarchy contain more detailed information about possible poses of some subpart of the body. In this article we describe our model and present experiments that show we can recover 3D human figures from 2D images in a view independent manner, and also track people the system has not been trained on.     

Human Tracking System Integrating Sound and Face Localization Using an Expectation-Maximization Algorithm in Real Environments

We have developed a human tracking system for use by robots that integrate sound and face localization. Conventional systems usually require many microphones and/or prior information to localize several sound sources. Moreover, they are incapable of coping with various types of background noise. Our system, the cross-power spectrum phase analysis of sound signals obtained with only two microphones, is used to localize the sound source without having to use prior information such as impulse response data. An expectation-maximization (EM) algorithm is used to help the system cope with several moving sound sources. The problem of distinguishing whether sounds are coming from the front or back is also solved with only two microphones by rotating the robot's head. A developed method that uses facial skin colors classified by another EM algorithm enables the system to detect faces in various poses. It can compensate for the error in the sound localization for a speaker and also identify noise signals entering from undesired directions by detecting a human face. A developed probability-based method is used to integrate the auditory and visual information in order to produce a reliable tracking path in real-time. Experiments using a robot showed that our system can localize two sounds at the same time and track a communication partner while dealing with various types of background noise.     

A Bayesian plan-view map based approach for multiple-person detection and tracking

This work proposes a novel approach for people detection and tracking in colour-with-depth sequences using a particle filtering approach. Detection and tracking are performed in plan-view maps integrating occupancy and height information with a novel plan-view map representation for colour information. Using the three maps, we propose a multiple particle filtering algorithm for people detection and tracking. The observation model proposed integrates information from the three maps so that people with different coloured clothes are not confused even when they interact at close distances. To avoid the coalescence problem when people with similar coloured clothes approach each other, the weight of particles is modified by an interaction factor that combines colour and position information. The algorithm also avoids the coalescence problem in case of total occlusion by means of an occlusion detection and recovering mechanism. Finally, a solution is proposed to improve the exponential complexity of multiple particle filters so that the algorithm proposed has linear complexity.The approach proposed has been tested in several colour-with-depth sequences where people move and interact freely in the environment. In the sequences, people walk at different distances, cross their paths causing frequent occlusions, jump, run and have close interactions such as shaking hands or embracing each other. The experimental results show that our proposal is able to detect and keep track of every person with a low error and deals with partial and total occlusions. Besides, the detection and tracking techniques presented are appropriate for large tracking problems in real-time applications since their complexity is linear, are suitable for parallel processing and allow the integration of information provided by multiple stereo vision sensors.     

An efficient camera calibration method for vision-based head tracking

The aim of this study is to develop and evaluate an efficient camera calibration method for vision-based head tracking. Tracking head movements is important in the design of an eye-controlled human/computer interface. A vision-based head tracking system is proposed to allow the user's head movements in the design of the eye-controlled human/computer interface. We propose an efficient camera calibration method to track the three-dimensional position and orientation of the user's head accurately. We also evaluate the performance of the proposed method and the influence of the configuration of calibration points on the performance. The experimental error analysis results showed that the proposed method can provide more accurate and stable pose (i.e. position and orientation) of the camera than the direct linear transformation method which has been used in camera calibration. The results for this study can be applied to the tracking of head movements related to the eye-controlled human/computer interface and the virtual reality technology.     

Information filtering for mobile augmented reality

The use of augmented reality (AR) techniques can revolutionize the way people interact with unfamiliar environments. By tracking the user's position and orientation, complicated spatial information can be registered against the real world. My colleagues and I are researching the problem of developing mobile AR systems to be worn by individual users operating in large, complicated environments such as cities. However, an urban environment is extremely complicated. It is populated by large numbers of buildings, each of which can have numerous facts stored about it. Therefore, it is easy for a user to experience information overload. This problem is illustrated. To minimize problems of information overload, we have begun to develop algorithms for information filtering. These tools automatically restrict the amount of information displayed     

人体自由运动状态下的视线追踪算法研究*

针对人体在大空间范围内自由运动时视线方向难以追踪的问题,构建了一套基于光学跟踪设备的头戴式视线追踪系统。系统通过被动式光学追踪设备和头戴式眼部摄像机获取使用者的头部运动状态与眼部图像,然后依据初始标定结果来估计使用者自由运动状态下的视线方向;最后对系统进行简化,得到了适用于同类环境、与具体硬件设备无关的视线跟踪三点三面三变换几何模型。对系统进行应用实验和误差分析表明,使用者在3.0 * 3.2 * 2.0 m的大工作空间内自由运动时视线追踪误差为1.69度,频率为20赫兹。     

基于图像序列的人体跟踪

由于人体的非刚体性和人体之间会经常发生遮挡,使得人体跟踪是一个很有挑战性的课题.针对这一特点,提出了用结合卡尔曼滤波和贝叶斯的方法来完成多个人体的跟踪,先建立简单的背景模型,然后用背景差分法得到前景区域,提取运动人体,并用EM(期望-最大化)算法建立相应的人体模型.在人体间没有发生遮挡时,用卡尔曼滤波方法来跟踪各个人体;人体间出现遮挡时,用贝叶斯方法来判别和跟踪相应人体.实验表明,该方法既能保证跟踪的快速性,又能很好地处理人体间相互遮挡的情况,该算法鲁棒性好,跟踪结果令人满意.     

Asymptotic moving object tracking with trajectory tracking extension: A homography‐based approach

In this paper, a homography‐based visual servo controller is developed for a rigid body to track a moving object in three‐dimensional space with a fixed relative pose. Specifically, a monocular camera is mounted on the rigid body, and the desired relative pose is expressed by a pre‐recorded reference image. Homography is exploited to obtain the orientation and scaled position for controller design. Considering the unknown moving object's velocities and distance information, a continuous nonlinear visual controller is developed using the robust integral of the signum of the error methodology. To facilitate the stability analysis, the system uncertainties regarding the moving object's velocities and distance information are divided into the error‐unrelated system uncertainties and the error‐related system uncertainties. After that, the upper bounds of the error‐related system uncertainties are derived with composited system errors. An asymptotic tracking of the leading object is proved based on the Lyapunov methods and the derived upper bounds. In addition, the proposed controller is extended to address the trajectory tracking problem. Simulation results validate the effectiveness of the proposed approach. Copyright © 2017 John Wiley & Sons, Ltd.     

Markov chain Monte Carlo based video tracking algorithm

The paper considers a problem of multiple person tracking. We present the algorithm to automatic people tracking on surveillance videos recorded by static cameras. Proposed algorithm is an extension of approach based on tracking-by-detection of people heads and data association using Markov chain Monte Carlo (MCMC). Short track fragments (tracklets) are built by local tracking of people heads. Tracklet postprocessing and accurate results interpolation were shown to reduce number of false positives. We use position deviations of tracklets and revised entry/exit points factor to separate pedestrians from false positives. The paper presents a new method to estimate body position, that increases precision of tracker. Finally, we switched HOG-based detector to cascade one. Our evaluation shows proposed modifications significantly increase tracking quality.     

Robust consensus tracking of second-order nonlinear systems using relative position information by K-filter and disturbance observer based control

This work considers the problem of distributed consensus tracking control of second-order uncertain nonlinear systems under a directed communication graph which contains a spanning tree, where the leader node is the root. It is assumed that the followers receive only the relative positions from the neighbours. For the purpose of consensus tracking controller design, in each follower, a group of K-filters is introduced so that the necessity of velocity estimating is avoided. Then we can express each follower's tracking error dynamics as a second-order system with mismatched uncertainties. And hence we can design a robust consensus tracking controller for each follower by using the combination of the backstepping design and the disturbance observer based control using only relative position information. Theoretical analysis is performed to show that the DOBs' estimation errors can be made to decay to be sufficiently small very quickly before the system states escape from the feasible region. Then we show that all the followers' states track those of the leader with arbitrarily small ultimate error bounds. And simulation examples are provided to demonstrate the performance of the proposed method.     

Biological cell injection visual and haptic interface

This paper presents a new three-dimensional (3-D) biomicromanipulation system for biological objects such as embryos, cells or oocytes. As the cell is very small, kept in liquid and observed through a microscope, 2-D visual feedback makes accurate manipulation in the 3-D world difficult. To improve the manipulation work, we proposed an intelligent human–machine interface. The 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to perform stable microinjection tasks, the operator needs force feedback and haptic assistance during penetration of the cell envelop — the chorion. Thus, realistic haptic rendering techniques have been implemented to validate stable insertion of a micropipette in a living cell. The proposed human–machine user's interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates, virtual haptic rendering to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by adequately controlling position, velocity and force parameters. Experiments showed that the virtualized reality interface acts as a tool for total guidance and assistance during microinjection tasks.     

People counting via multiple views using a fast information fusion approach

Real-time estimates of a crowd size is a central task in civilian surveillance. In this paper we present a novel system counting people in a crowd scene with overlapping cameras. This system fuses all single view foreground information to localize each person present on the scene. The purpose of our fusion strategy is to use the foreground pixels of each single views to improve real-time objects association between each camera of the network. The foreground pixels are obtained by using an algorithm based on codebook. In this work, we aggregate the resulting silhouettes over cameras network, and compute a planar homography projection of each camera’s visual hull into ground plane. The visual hull is obtained by finding the convex hull of the foreground pixels. After the projection into the ground plane, we fuse the obtained polygons by using the geometric properties of the scene and on the quality of each camera detection. We also suggest a region-based approach tracking strategy which keeps track of people movements and of their identities along time, also enabling tolerance to occasional misdetections. This tracking strategy is implemented on the result of the views fusion and allows to estimate the crowd size dependently on each frame. Assessment of experiments using public datasets proposed for the evaluation of counting people system demonstrates the performance of our fusion approach. These results prove that the fusion strategy can run in real-time and is efficient for making data association. We also prove that the combination of our fusion approach and the proposed tracking improve the people counting.

     

Scalable and robust multi-people head tracking by combining distributed multiple sensors

In this paper, we present a robust 3D human-head tracking method. 3D head positions are essential for robots interacting with people. Natural interaction behaviors such as making eye contacts require head positions. Past researches with laser range finder (LRF) have been successful in tracking 2D human position with high accuracy in real time. However, LRF trackers cannot track multiple 3D head positions. On the other hand, trackers with multi-viewpoint images can obtain 3D head position. However, vision-based trackers generally lack robustness and scalability, especially in open environments where lightening conditions vary by time. To achieve 3D robust real-time tracking, here we propose a new method that combines LRF tracker and multi-camera tracker. We combine the results from trackers using the LRF results as maintenance information toward multi-camera tracker. Through an experiment in a real environment, we show that our method outperforms toward existing methods, both in its robustness and scalability.     

基于头部永磁跟踪的计算机虚拟实现技术

根据永磁体在三维空间的特定分布规律设计了一种基于永磁跟踪的残疾人计算机辅助系统——一种跟踪头部运动的装置。使用时将三轴磁场传感器置于人头部,实时地接收永磁体的磁场信号,采用磁场跟踪算法,得到三轴磁场传感器相对于永磁体的空间位置和姿势,进而得到人头部的运动方向和角度,并将这些方向和角度作为控制信号,控制屏幕上的鼠标动作,帮助操作者通过人机交互界面实现电脑屏幕鼠标控制。     

Evaluation of realtime people tracking for indoor environments using ubiquitous motion sensors and limited wireless network infrastructure

We present the development and evaluation of a realtime indoor localisation system for tracking people. Our aim was to track a person’s indoor position using dead-reckoning, while limiting position error without depending on extensive wireless network infrastructure. The Indoor People Tracker used wearable motion sensors, a floor-plan map and a limited wireless sensor network for proximity ranging. We evaluated how the position accuracy of the Indoor People Tracker was affected by floor-plan map features, wireless proximity range and motion information. The advantage of the Indoor People Tracker was found; it was able to achieve accurate position resolution with minimal error, while not depending on wireless proximity.     

Trajectory prediction and tracking using a multi-behaviour social particle filter

3D motion tracking is a challenging task when both the tracked object and the observer are moving. In this paper, we present a multi-behavioural social force-based particle filter to track a group of moving humans from a moving robot using a limited field-of-view monocular camera. The application is a robotic guide and while moving, the robot often loses visibility of one or more people in the group, who must still be tracked. As an example, due to limited space, when the robot takes a sharp turn to avoid an obstacle or circumvent a corner, the visibility of the people at the rear is lost for some time. Therefore, several human social behavioural aspects have been implemented to predict the human’s motion in a group. The model accounts for attraction and repulsion between the people of the group and those with the robot, to maintain a comfortable social distance with each other at equilibrium. Additionally, when any person leaves the group then the track is deleted and after joining the track is automatically re-initialized. In the literature, the time of invisibility is a criterion to detect a person who has left the system, which however cannot be used here since the invisibility may be due to a limited field of view or the robot making a sharp turn to avoid an obstacle or circumventing a corner. Social heuristics are used to accurately detect people leaving the robotic system. The tracked trajectory is compared with ground truth and our system gives a very less error when compared with several baseline approaches. False positives are reduced, and the accuracy also increased with our proposed model as compared to other baseline methods. This method has been tested on several scenarios to ensure its validity.

     

Ambivalent effect of member portraits in virtual groups

Abstract Knowledge exchange with shared databases can be seen as a public-goods dilemma. People are reluctant to contribute information because they save time, effort and perhaps social power if they withhold their knowledge and socially loaf. But if all people choose this individually efficient strategy, then no information exchange can take place, and the group is less effective than it would have been if all members contributed. Thus, in the knowledge-exchange situation, group norms and individual norms oppose each other. In order to strengthen people's orientation towards the group, virtual collaborative tools sometimes provide member portraits. But based on the Social Identity Model of Deindividuation Effect (SIDE)-Model, ambivalent effects of these member portraits are expected according to people's social categorization: when people identify more strongly with the group, such portraits enhance participation, whereas when people have a stronger individual identity, they undermine participation. This study links the concepts of social value orientation to identity salience. The assumption is that in the information-exchange dilemma, the social value orientation of a group member will determine whether group identity or individual identity becomes salient for that group member. We then extend the SIDE-model to the domain of social loafing and expect that portraits of the group members will have different effects for people with individual orientation and for people with prosocial orientation. An experiment confirmed this expectation and revealed a significant interaction between social value orientation and portraits. Especially striking is the result that for prosocials the provision of member portraits leads to an increase in social loafing. This is analogous to the predictions of the SIDE-model.     

High-precision identification of contextual information in location-aware engineering applications

This paper presents research that investigated algorithms for high-precision identification of contextual information in location-aware engineering applications. The primary contribution of the presented work is the design and implementation of a dynamic user-viewpoint tracking scheme in which mobile users’ spatial context is defined not only by their position (i.e., location), but also by their three-dimensional head orientation (i.e., line of sight). This allows the identification of objects and artifacts visible in a mobile user’s field of view with much higher accuracy than was possible by tracking position alone. For outdoor applications, a georeferencing based algorithm has been developed using the Global Positioning System (GPS) and magnetic orientation tracking devices [5] to track a user’s dynamic viewpoint. For indoor applications, this study explored the applicability of wireless technologies, in particular Indoor GPS, for dynamic user position tracking in situations where GPS is unavailable. The objectives of this paper are to describe the details of the three-stage-algorithm that has been designed and implemented, and to demonstrate the extent to which positioning technologies such as GPS and Indoor GPS can be used together with high-precision orientation trackers to accurately interpret the fully-qualified spatial context of a mobile user in challenging environments such as those found on construction sites. The obtained results highlight the potential of using location-aware technologies for rapidly identifying and retrieving contextual information in engineering applications.     

视频中多特征融合人体姿态跟踪

目的 目前已有的人体姿态跟踪算法的跟踪精度仍有待提高,特别是对灵活运动的手臂部位的跟踪。为提高人体姿态的跟踪精度,本文首次提出一种将视觉时空信息与深度学习网络相结合的人体姿态跟踪方法。方法 在人体姿态跟踪过程中,利用视频时间信息计算出人体目标区域的运动信息,使用运动信息对人体部位姿态模型在帧间传递;考虑到基于图像空间特征的方法对形态较为固定的人体部位如躯干和头部能够较好地检测,而对手臂的检测效果较差,构造并训练一种轻量级的深度学习网络,用于生成人体手臂部位的附加候选样本;利用深度学习网络生成手臂特征一致性概率图,与视频空间信息结合计算得到最优部位姿态,并将各部位重组为完整人体姿态跟踪结果。结果 使用两个具有挑战性的人体姿态跟踪数据集VideoPose2.0和YouTubePose对本文算法进行验证,得到的手臂关节点平均跟踪精度分别为81.4%和84.5%,与现有方法相比有明显提高;此外,通过在VideoPose2.0数据集上的实验,验证了本文提出的对下臂附加采样的算法和手臂特征一致性计算的算法能够有效提高人体姿态关节点的跟踪精度。结论 提出的结合时空信息与深度学习网络的人体姿态跟踪方法能够有效提高人体姿态跟踪的精度,特别是对灵活运动的人体姿态下臂关节点的跟踪精度有显著提高。