Smartphone-based pedestrian tracking in indoor corridor environments

As the use of smartphones spreads rapidly, user localization becomes an important issue for providing diverse location-based services (LBS). While tracking users in outdoor environments is easily done with GPS, the solution for indoor tracking is not trivial. One common technique for indoor user tracking is to employ inertial sensors, but such a system needs to be capable of handling noisy sensors that would normally lead to cumulative locating errors. To reduce such error, additional infrastructure has often been deployed to adjust for these cumulative location errors. As well, previous work has used highly accurate sensors or sensors that are strapped to the body. This paper presents a stand-alone pedestrian tracking system, using only a magnetometer and an accelerometer in a smartphone in indoor corridor environments that are normally laid out in a perpendicular design. Our system provides reasonably accurate pedestrian locations without additional infrastructure or sensors. The experiment results show that the location error is less than approximately 7 m, which is considered adequate for indoor LBS applications.     

A Dynamic Gesture and Posture Recognition System

This paper presents a real time dynamic hand gesture and posture recognition system based on a neural network and a Hidden Markov Model. For skin color segmentation an adaptive online trained skin color model is used, while the hand posture recognition is accomplished through a likelihood-based classification technique of geometric features. A novel trajectory smoothing technique based on Self Organized Neural Network is introduced to improve HMM classification performance of dynamic gestures. The aim of the proposed system is the creation of a visual dictionary combining hand postures and dynamic gestures. The system has been successfully tested with many people under varying light conditions and different web cameras.     

Hand posture and gesture recognition technology

Hand gestures that are performed by one or two hands can be categorized according to their applications into different categories including conversational, controlling, manipulative and communicative gestures. Generally, hand gesture recognition aims to identify specific human gestures and use them to convey information. The process of hand gesture recognition composes mainly of four stages: hand gesture images collection, gesture image preprocessing using some techniques including edge detection, filtering and normalization, capture the main characteristics of the gesture images and the evaluation (or classification) stage where the image is classified to its corresponding gesture class. There are many methods that have been used in the classification stage of hand gesture recognition such as Artificial Neural Networks, template matching, Hidden Markov Models and Dynamic Time Warping. This exploratory survey aims to provide a progress report on hand posture and gesture recognition technology.     

Learning object,grasping and manipulation activities using hierarchical HMMs

This article presents a probabilistic algorithm for representing and learning complex manipulation activities performed by humans in everyday life. The work builds on the multi-level Hierarchical Hidden Markov Model (HHMM) framework which allows decomposition of longer-term complex manipulation activities into layers of abstraction whereby the building blocks can be represented by simpler action modules called action primitives. This way, human task knowledge can be synthesised in a compact, effective representation suitable, for instance, to be subsequently transferred to a robot for imitation. The main contribution is the use of a robust framework capable of dealing with the uncertainty or incomplete data inherent to these activities, and the ability to represent behaviours at multiple levels of abstraction for enhanced task generalisation. Activity data from 3D video sequencing of human manipulation of different objects handled in everyday life is used for evaluation. A comparison with a mixed generative-discriminative hybrid model HHMM/SVM (support vector machine) is also presented to add rigour in highlighting the benefit of the proposed approach against comparable state of the art techniques.     

Speech driven photo realistic facial animation based on an articulatory DBN model and AAM features

This paper presents a photo realistic facial animation synthesis approach based on an audio visual articulatory dynamic Bayesian network model (AF_AVDBN), in which the maximum asynchronies between the articulatory features, such as lips, tongue and glottis/velum, can be controlled. Perceptual Linear Prediction (PLP) features from audio speech, as well as active appearance model (AAM) features from face images of an audio visual continuous speech database, are adopted to train the AF_AVDBN model parameters. Based on the trained model, given an input audio speech, the optimal AAM visual features are estimated via a maximum likelihood estimation (MLE) criterion, which are then used to construct face images for the animation. In our experiments, facial animations are synthesized for 20 continuous audio speech sentences, using the proposed AF_AVDBN model, as well as the state-of-art methods, being the audio visual state synchronous DBN model (SS_DBN) implementing a multi-stream Hidden Markov Model, and the state asynchronous DBN model (SA_DBN). Objective evaluations on the learned AAM features show that much more accurate visual features can be learned from the AF_AVDBN model. Subjective evaluations show that the synthesized facial animations using AF_AVDBN are better than those using the state based SA_DBN and SS_DBN models, in the overall naturalness and matching accuracy of the mouth movements to the speech content.     

Automatic calibration of laser range finder positions for pedestrian tracking based on social group detections

Laser range finders (LRF’s) are non-invasive sensors which can be used for high-precision and anonymous tracking of pedestrians in social environments. Such sensor networks can be used in robotics to assist in navigation and human–robot interaction. Typically, multiple LRF’s are used together for such tasks, and the relative positions of these sensors must be precisely calibrated. We propose a technique for estimating relative LRF positions using observations of social groups in the pedestrian flow as keypoint features for determining coarse estimates of relative sensor offsets. The most likely offset is estimated using a generalized Hough transform and used to identify sets of possible shared observations of individual pedestrians between pairs of sensors. Outliers are rejected using the RANSAC technique, and the resulting shared observations from each sensor pair are combined into a constraint matrix for the sensor network, which is solved using least-squares minimization. Results show calibration accuracy of sensor positions within 34?mm and 0.51°, and an analysis of pedestrian data collected from ubiquitous networks in three public and commercial spaces shows that the proposed calibration technique enables pedestrian tracking within 11?cm accuracy.     

基于神经网络的多传感器融合PDR定位方法

针对当前行人航位推算系统因行人随意性行走、传感器漂移等造成行人步长估计不精确、方向计算误差累积问题,提出了一种基于神经网络和智能手机内置多传感器融合的PDR室内定位方法.首先利用加速计采集的传感器数据和移动距离数据训练BP神经网络,将训练好的BP神经网络模型进行行人移动距离预测,然后根据行人行走步伐的连续性特点和传感器输出之间的相关性,设计了一种微航向角融合的方向估计算法.该算法通过对行走过程中的情况进行分类以获得可靠的传感器源,利用3种微航向角进行分类加权融合,最终获得行人行走方向的精确估计.实验结果表明,通过行人移动距离预测和微航向角融合算法能够实现得较好的定位效果.     

A 3D indoor positioning system based on low-cost MEMS sensors

A positioning system in the absence of GPS is important in establishing indoor directional guidance and localization. Inertial Measuring Units (IMUs) can be used to detect the movement of a pedestrian. In this paper, we present a three-dimensional (3D) indoor positioning system using foot mounted low cost Micro-Electro-Mechanical System (MEMS) sensors to locate the position and attitude of a person in 3D view, and to plot the path travelled by the person. The sensors include accelerometers, gyroscopes, and a barometer. The pedestrians motion information is collected by accelerometers and gyroscopes to achieve Pedestrian Dead-Reckoning (PDR) which is used to estimate the pedestrian’s rough position. A zero velocity update (ZUPT) algorithm is developed to detect the standing still moment. A Kalman filter is combined with the ZUPT to eliminate non-linear errors in order to obtain accurate positioning information of a pedestrian. The information collected by the barometer is integrated with the accelerometer data to detect the altitude changes and to obtain accurate height information. The main contribution of this research is that the approach proposed fuses barometer and accelerometer in Kalman filter to obtain accurate height information, which has improved the accuracy at x axis and y axis. The proposed system has been tested in several simulated scenarios and real environments. The distance errors are around 1%, and the positioning errors are less than 1% of the total travelled distance. Results indicate that the proposed system performs better than other similar systems using the same low-cost IMUs.     

Pedestrian detection and tracking at crossroads

This paper presents a system that can perform pedestrian detection and tracking using vision-based techniques. A very important issue in the field of intelligent transportation system is to prevent pedestrians from being hit by vehicles. Recently, a great number of vision-based techniques have been proposed for this purpose. In this paper, we propose a vision-based method, which combines the use of a pedestrian model as well as the walking rhythm of pedestrians to detect and track walking pedestrians. Through integrating some spatial and temporal information grabbed by a vision system, we are able to develop a reliable system that can be used to prevent traffic accidents happened at crossroads. In addition, the proposed system can deal with the occlusion problem. Experimental results obtained by executing some real world cases have demonstrated that the proposed system is indeed superb.     

基于行人运动模态辨识的室内外无缝导航航向算法研究

目前行人导航航向解算算法均基于导航传感器在行人身体上的固定安装模式,或者依赖其他射频信息辅助修正陀螺航向,这极大约束了导航传感器的适用条件.为此,利用陀螺对低频噪声的敏感性及加速度计低频的稳定性,提出了解决行人手持手机稳态查看与非稳态摇摆的运动模态辨识算法和基于时域互补滤波器实现姿态的最优融合方法;研究了改进型互补滤波以消除行人的运动加速度对姿态解算的干扰误差,提高了载体姿态的测量精度;此外,利用磁传感器标定后的数据设计了自适应卡尔曼滤波算法,抑制了航向角的误差发散.经实际数据测试验证,室内外行人手持稳态与非稳态下的航向角测量精度提高了80%,同时大大提高了导航传感器的适用性与便携性,满足实际工程的使用需求.     

基于OTSU分割和HOG特征的行人检测与跟踪方法

传统的HOG算法针对整幅图像进行行人特征提取,大量的非人窗口计算必然降低检测的准确率和效率。为此,提出一种基于OTSU分割和HOG特征的行人检测与跟踪方法。利用OTSU算法以最佳阈值分割图像,在分割区域的基础上进行Canny边缘检测,通过边缘的对称性计算确定行人候选区,继而采用经PCA方法降维后的HOG特征和隐马尔可夫模型对行人候选区进行检测验证。最后,以确定的行人区域为跟踪窗口,利用CamShift算法跟踪行人。多组实验结果证明,本文方法的行人检测效率和精度均有所提高,跟踪性能稳定、可靠。     

基于似然域背景差分的行人检测和跟踪方法

基于激光雷达的室内机器人行人检测、跟踪容易受到复杂背景的影响。针对这种情况,提出一种基于似然域背景差分的行人检测、跟踪和跟随系统。利用即时定位与地图构建算法获得陌生环境的二维栅格地图,通过蒙特卡洛定位获得机器人在地图中的后验位姿,利用似然域模型分割出前景对应的激光雷达数据后,进行行人的检测、跟踪以及跟随。实验结果表明,该系统使行人检测准确率提升3.49%,平均检测时间缩短近32%,有效降低复杂背景对多行人检测与跟踪的影响,实现机器人对目标行人的实时跟随。     

基于步行加速度信息分割的人员识别

为提高基于智能手机内置加速度传感器的人员识别率,提出了一种基于信息分割的组合分类器识别方法。根据人员步行加速度变化特点提出了基于HMM（隐马尔可夫模型）的划分方法,将人员步行加速度划分成相对动态与稳态两个部分,分别从两个区域提取标准差、均值、能量等特征;根据不同步行速率选择这些特征和峰值点连线斜率组合成新的特征集合;最后,采用组合分类器的方法获得了更加理想的识别精度。实验结果表明,在人员慢步行走的姿态下的识别率达到了98.3%,快速步行达到了97.6%。较现有人员识别方法有较大的提高。     

基于GLRT零速检测算法的行人室内定位系统

针对微机电系统中惯性传感器漂移大、精度低导致室内行人定位精度不高的问题,本系统在惯性导航解算算法的基础上,提出基于广义似然比检验的零速检测算法.该方法是利用广义似然比检验对行人处于站立相或摆动相的概率进行估计以及进行零速更新,提高行人定位精度.基于本文提出的行人室内定位模型,搭建以惯性测量单元为核心的实验平台,评估本文算法的可行性.实验结果表明行人定位的动态误差为-1.8141 m~1.4516 m,置信度为97.61%.表明本文的行人室内定位系统满足实际定位的要求.     

Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes

Tracking pedestrians is a vital component of many computer vision applications, including surveillance, scene understanding, and behavior analysis. Videos of crowded scenes present significant challenges to tracking due to the large number of pedestrians and the frequent partial occlusions that they produce. The movement of each pedestrian, however, contributes to the overall crowd motion (i.e., the collective motions of the scene's constituents over the entire video) that exhibits an underlying spatially and temporally varying structured pattern. In this paper, we present a novel Bayesian framework for tracking pedestrians in videos of crowded scenes using a space-time model of the crowd motion. We represent the crowd motion with a collection of hidden Markov models trained on local spatio-temporal motion patterns, i.e., the motion patterns exhibited by pedestrians as they move through local space-time regions of the video. Using this unique representation, we predict the next local spatio-temporal motion pattern a tracked pedestrian will exhibit based on the observed frames of the video. We then use this prediction as a prior for tracking the movement of an individual in videos of extremely crowded scenes. We show that our approach of leveraging the crowd motion enables tracking in videos of complex scenes that present unique difficulty to other approaches.     

Pedestrian tracking using color, thermal and location cue measurements: a DSmT-based framework

In this paper, we address the problem of pedestrians tracking in cluttered scenes using location, color and thermal cues. The Dezert–Smarandache (DSm) theoretical framework is used to combine the measurements provided by the sensors into a single and unified frame. The use of DSm Theory allows modeling the conflicts that might arise between the sensors due to the presence of clutter and high levels of occlusion. The location cue is integrated as a prior knowledge, which increases the robustness of the tracking. A belief measure is derived and used as a step in a particle filtering algorithm. Finally, experimental results are given, where the developed approach is used to track walking persons in indoor scenes with high levels of occlusion and clutter.     

Prediction of pedestrians routes within a built environment in normal conditions

Modelling and prediction of pedestrian routing behaviours within known built environments has recently attracted the attention of researchers across multiple disciplines, owing to the growing demand on urban resources and requirements for efficient use of public facilities. This study presents an investigation into pedestrians’ routing behaviours within an indoor environment under normal, non-panic situations. A network-based method using constrained Delaunay triangulation is adopted, and a utility-based model employing dynamic programming is developed. The main contribution of this study is the formulation of an appropriate utility function that allows an effective application of dynamic programming to predict a series of consecutive waypoints within a built environment. The aim is to generate accurate sequence waypoints for the pedestrian walking path using only structural definitions of the environment as defined in a standard CAD format. The simulation results are benchmarked against those from the A¹ algorithm, and the outcome positively indicates the usefulness of the proposed method in predicting pedestrians’ route selection activities.     

基于位置纠正和Kalman滤波算法的目标跟踪方法

本文基于判别尺度空间跟踪算法,将位置纠正方法和卡尔曼滤波算法应用于行人跟踪中.为解决行人因形变和环境变化导致的跟踪不准确的问题,本文充分利用fhog特征在行人跟踪上的优势,以判别尺度空间算法中的位置滤波器所计算的位置为中心,再次提取行人的fhog特征并将其与位置滤波器模板做相关运算,以此纠正行人位置.其次,利用卡尔曼滤波算法对纠正后的行人位置进行预测和再次纠正,最终在双重纠正的位置上训练新的位置滤波器模板.本文选取OTB-100中的行人数据集对该方法进行性能测试,实验结果表明,在原算法位置上,再次提取fhog特征进行相关运算能够纠正行人的位置,同时卡尔曼滤波对纠正位置进行预测和再次纠正,可使行人的定位精度再次提升.     

基于视野域机制的行人轨迹预测

为提高行人在复杂交通场景中交互的安全性,提出一种基于social-GAN（social-generative adversarial network）的行人轨迹预测算法SAN-GAN（social angle norm-GAN）。该算法首先以行人历史位置信息与头部信息为输入,通过轨迹生成器LSTM网络（long short term memory networks）获取行人隐藏特征信息,并基于行人视野域模块捕捉行人视野域动态变化,对所有行人建立扇形视野域并筛选有效信息,从而驱动神经网络模型预测行人未来轨迹变化。将SAN-GAN与LSTM、social-LSTM（social-long short term memory networks）、social-GAN等轨迹预测算法进行对比实验,结果表明SAN-GAN算法相较于其他算法,在预测3.2 s的行人轨迹时,ADE分别平均降低65.8%、51.2%、10.7%,FDE分别平均降低73.6%、60.9%、10.4%。SAN-GAN能够有效地预测行人在复杂交通环境中进行交互的未来轨迹。     

基于道路环境上下文的行人跟踪方法

针对目前城市交通中人车混行场景中行人跟踪效果不佳的问题,提出了一种基于道路环境上下文的行人跟踪方法。首先通过对道路环境上下文进行分析,建立道路模型;其次在道路模型的约束下建立行人与环境的交互运动模型;最后利用该模型进行行人的跟踪。在真实场景中的实验表明使用了道路上下文信息的跟踪方法与连续离散连续能量最小化的多行人跟踪方法相比,多目标跟踪准确度从47.6%提升至63.2%,多目标跟踪精度从68.8%提升至74.3%。数值结果表明道路上下文信息对于提高人车混行场景中行人跟踪效果的有效性。