Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes

Tracking pedestrians is a vital component of many computer vision applications, including surveillance, scene understanding, and behavior analysis. Videos of crowded scenes present significant challenges to tracking due to the large number of pedestrians and the frequent partial occlusions that they produce. The movement of each pedestrian, however, contributes to the overall crowd motion (i.e., the collective motions of the scene's constituents over the entire video) that exhibits an underlying spatially and temporally varying structured pattern. In this paper, we present a novel Bayesian framework for tracking pedestrians in videos of crowded scenes using a space-time model of the crowd motion. We represent the crowd motion with a collection of hidden Markov models trained on local spatio-temporal motion patterns, i.e., the motion patterns exhibited by pedestrians as they move through local space-time regions of the video. Using this unique representation, we predict the next local spatio-temporal motion pattern a tracked pedestrian will exhibit based on the observed frames of the video. We then use this prediction as a prior for tracking the movement of an individual in videos of extremely crowded scenes. We show that our approach of leveraging the crowd motion enables tracking in videos of complex scenes that present unique difficulty to other approaches.     

A social force evacuation model driven by video data

This paper proposes a video data-driven social force model for simulating crowd evacuation. The initialization of pedestrian position, path navigation, and goal selection in the improved social force model was guided by real video data. To initialize pedestrian position and determine path navigation, the distribution of the pedestrians is set according to the real video. We also extracted the trajectories of pedestrian movement from the videos, and these trajectories were stored into a path set to guide the evacuation of pedestrians. Moreover, a fitness function was defined to model the behavior of a pedestrian goal selection. The fitness function could process the evacuation parameters, which were extracted from the video, and consider the degree and distance of exit congestion. Furthermore, we quantified the relationship values among pedestrians, and a new force called “group force” was added to the primary social force model. Pedestrians with close relationship gathered into one group and walked together. To validate the effectiveness of the proposed method, the video data-driven model was applied to simulate campus halls and roads. Simulation results show that the proposed approach is consistent with real-world situations and can assist in analyzing emergency evacuation scenarios.     

A cognitive pedestrian behavior model for exploratory navigation: Visibility graph based heuristics approach

Navigation is a complex issue in simulating pedestrian dynamics. Many existing studies have investigated how pedestrians navigate from an origin to a specific destination, i.e., traveling to familiar or novel destinations. However, pedestrians’ navigation through built environment without specific destinations still remains as an open issue. This exploratory navigation usually involves more spatial cognitive behaviors and perceptual considerations. To represent realistic route choice and natural movement of theses pedestrians, this paper presents a cognitive pedestrian behavior model with a focus on the space visibility and individual characteristics. Then the sensitivity of model parameters is analyzed, indicating that regional target, memory of visiting records and individual crowd tendency can have varied influences on pedestrians’ navigation decisions and movement patterns. As a case study, the proposed model is implemented using the pedestrian movement data collected from an observational study of Tate Gallery Museum. The simulation results show good consistency with the actual data at the aggregate level and those obtained individually, indicating that our proposed model is credible and can benefit real applications such as master planning of a museum.     

Visual Motion Segmentation in Crowd Videos Based on Spatial-Angular Stacked Sparse Autoencoders

Visual motion segmentation (VMS) is an important and key part of many intelligent crowd systems. It can be used to figure out the flow behavior through a crowd and to spot unusual life-threatening incidents like crowd stampedes and crashes, which pose a serious risk to public safety and have resulted in numerous fatalities over the past few decades. Trajectory clustering has become one of the most popular methods in VMS. However, complex data, such as a large number of samples and parameters, makes it difficult for trajectory clustering to work well with accurate motion segmentation results. This study introduces a spatial-angular stacked sparse autoencoder model (SA-SSAE) with l2-regularization and softmax, a powerful deep learning method for visual motion segmentation to cluster similar motion patterns that belong to the same cluster. The proposed model can extract meaningful high-level features using only spatial-angular features obtained from refined tracklets (a.k.a ‘trajectories’). We adopt l2-regularization and sparsity regularization, which can learn sparse representations of features, to guarantee the sparsity of the autoencoders. We employ the softmax layer to map the data points into accurate cluster representations. One of the best advantages of the SA-SSAE framework is it can manage VMS even when individuals move around randomly. This framework helps cluster the motion patterns effectively with higher accuracy. We put forward a new dataset with its manual ground truth, including 21 crowd videos. Experiments conducted on two crowd benchmarks demonstrate that the proposed model can more accurately group trajectories than the traditional clustering approaches used in previous studies. The proposed SA-SSAE framework achieved a 0.11 improvement in accuracy and a 0.13 improvement in the F-measure compared with the best current method using the CUHK dataset.     

结合最短路径改进的社会力人群疏散仿真模型

社会力模型广泛应用于人群疏散仿真,针对该模型在仿真过程中存在行人停滞不前、无法通过非凸边形障碍物和疏散路径与行人实际选择的路径不相符等问题,提出了一种社会力改进模型。该模型基于场景中的障碍物生成路径节点,利用这些节点生成无向图,同时考虑了节点的安全系数和拥挤系数对节点通行性的影响生成最短疏散路径。通过改进后的社会力模型进行了多种场景的仿真实验,实验结果显示行人在复杂障碍物场景中能有效绕过障碍物,生成合理的疏散路径,表明该模型有效改善社会力模型,使人群疏散仿真更加真实。     

生物启发的人群突发局部聚集感知神经网络

公共场所中的人群突发局部聚集常是异常事件发生的先兆,由于其随机性强,前兆特征不明显,现有的传统计算机视觉技术较难对其有效检测。基于蝗虫视觉系统的神经结构特性与小叶巨型运动检测器（lobula giant movement detector,LGMD）危险感知机理,提出一种人群突发局部聚集行为检测的LGMD改进型神经网络模型。该模型感知人群活动在视野域中引发的视觉信号,基于哺乳动物视网膜视觉信号处理机制整合视觉运动线索,借助LGMD神经元危险感知机理构建尖峰阈值机制调谐神经网络输出,以感知人群活动中的突发聚集行为。不同场景下的人群活动视频实验结果表明,提出的神经网络能有效检测视野域中人群突发局部聚集行为并对其预警。该文涉及生物视神经机理启发的人群活动动态视觉信息加工处理,可为智能视频监控中的人群活动检测与行为分析提供新思想、新方法。     

Learning motion patterns in unstructured scene based on latent structural information

ContextAs trajectory analysis is widely used in the fields of video surveillance, crowd monitoring, behavioral prediction, and anomaly detection, finding motion patterns is a fundamental task for pedestrian trajectory analysis.ObjectiveIn this paper, we focus on learning dominant motion patterns in unstructured scene.MethodsAs the invisible implicit indicator to scene structure, latent structural information is first defined and learned by clustering source/sink points using CURE algorithm. Considering the basic assumption that most pedestrians would find the similar paths to pass through an unstructured scene if their entry and exit areas are fixed, trajectories are then grouped based on the latent structural information. Finally, the motion patterns are learned for each group, which are characterized by a series of statistical temporal and spatial properties including length, duration and envelopes in polar coordinate space.ResultsExperimental results demonstrate the feasibility and effectiveness of our method, and the learned motion patterns can efficiently describe the statistical spatiotemporal models of the typical pedestrian behaviors in a real scene. Based on the learned motion patterns, abnormal or suspicious trajectories are detected.ConclusionThe performance of our approach shows high spatial accuracy and low computational cost.     

基于地面场粒子群优化算法的高密度人群应急疏散建模

针对非常规突发事件环境下高密度人群的拥挤管理和快速疏散问题,提出一种由感知层、传输层、计算层和应用层构成的多层结构人群疏散信息物理系统（E-CPS）体系框架。在E-CPS体系框架计算层中将静态地面场（FF）建模规则引入经典粒子群优化（PSO）模型,提出地面场PSO （FF-PSO）人群疏散模型,该模型同时具备静态场规则简单、计算快和PSO模型快速搜索、快速收敛的优点。此外,FF-PSO模型中构建了一种新的适应度函数,实现了疏散策略的动态选择,并通过数值仿真及实例仿真验证了FF-PSO模型在拥挤管理中的可行性和有效性。国家会展中心（上海）的实例仿真结果表明,考虑拥堵管理比仅考虑距离最短平均每分钟可多疏散66人,疏散时间节省19 min,疏散效率提升13.4%。     

人群应急疏散可视仿真研究进展和问题

人群应急疏散可视仿真是用智能体来模拟具有自主感知、情绪和行为能力的人群个体,并采用3维可视的方式来直观呈现人群应急疏散情景,可以为制定人群应急预案提供形象直观的分析方法。本文从人群仿真数据的来源、人群导航模型的构建、人群行为模型、人群情绪感染、人群渲染5个方面概述目前研究的进展,然后从仿真模型的可验证性、人群疏散导航模型的构建、人与环境的物理模型、动物逃生实验与仿真、疏散中的社会行为表现以及人群情绪的可视计算6个角度讨论需要进一步研究的问题。针对需要深入研究的问题,指出借助于紧急事件的视频监控分析和虚拟人群情景的用户调查,有助于完善人群仿真模型。结合物理模型,可以更准确地描述人群应急疏散场景。开展动物逃生实验分析,有助于完善人群运动导航算法。建立人群社会行为模型,可以更详细描述疏散中人群行为的多样性。构建基于多通道感知的人群情绪感染计算方法,可以详尽描述情绪感染的过程。人群应急疏散行为的可视仿真研究在城市的安全管理方面具有重要的应用前景,但其研究仍存在很多亟待解决的问题,综合地运用多学科知识,完善实验手段是进一步推动研究的关键所在。     

CLUST: Simulating Realistic Crowd Behaviour by Mining Pattern from Crowd Videos

In this paper, we present a data‐driven approach to simulate realistic locomotion of virtual pedestrians. We focus on simulating low‐level pedestrians' motion, where a pedestrian's motion is mainly affected by other pedestrians and static obstacles nearby, and the preferred velocities of agents (direction and speed) are obtained from higher level path planning models. Before the simulation, collision avoidance processes (i.e. examples) are extracted from videos to describe how pedestrians avoid collisions, which are then clustered using hierarchical clustering algorithm with a novel distance function to find similar patterns of pedestrians' collision avoidance behaviours. During the simulation, at each time step, the perceived state of each agent is classified into one cluster using a neural network trained before the simulation. A sequence of velocity vectors, representing the agent's future motion, is selected among the examples corresponding to the chosen cluster. The proposed CLUST model is trained and applied to different real‐world datasets to evaluate its generality and effectiveness both qualitatively and quantitatively. The simulation results demonstrate that the proposed model can generate realistic crowd behaviours with comparable computational cost.     

人群应急疏散中一种多智能体情绪感染仿真模型

为了从情绪的视角分析紧急情境下人群的疏散行为,梳理了现有情绪感染的研究工作,总结了人群紧急状况下行为特点.采用智能体描述人群个体,提出一种多智能体情绪感染模型.其主体框架分为感知层、情绪层、感染层、行为层和行动层.归纳了产生情绪感染现象的3个条件及情绪感染的3个规则,提出了情绪感染的算法,考虑个体的个性和个体间距离因素,采用情绪强度和人群紧密度来计算个体疏散速度.用C#语言编制了仿真实验,采用真实的地震疏散案例,验证了仿真疏散时间和实际观测的基本一致.通过与以往基于传染病思路的情绪感染模型对比,所提出的模型可以更好地描述情绪感染从局部到整体的过程.实验结果表明,所提出的模型可以推演情绪驱动下的群体聚集行为,有望为制定应急疏散预案提供一种可视化分析方法.     

Trajectory learning for activity understanding: unsupervised, multilevel, and long-term adaptive approach

Society is rapidly accepting the use of video cameras in many new and varied locations, but effective methods to utilize and manage the massive resulting amounts of visual data are only slowly developing. This paper presents a framework for live video analysis in which the behaviors of surveillance subjects are described using a vocabulary learned from recurrent motion patterns, for real-time characterization and prediction of future activities, as well as the detection of abnormalities. The repetitive nature of object trajectories is utilized to automatically build activity models in a 3-stage hierarchical learning process. Interesting nodes are learned through Gaussian mixture modeling, connecting routes formed through trajectory clustering, and spatio-temporal dynamics of activities probabilistically encoded using hidden Markov models. Activity models are adapted to small temporal variations in an online fashion using maximum likelihood regression and new behaviors are discovered from a periodic retraining for long-term monitoring. Extensive evaluation on various data sets, typically missing from other work, demonstrates the efficacy and generality of the proposed framework for surveillance-based activity analysis.     

Optimal crowd editing

Simulating realistic crowd behaviors is a challenging problem in computer graphics. Yet, several satisfying simulation models exhibiting natural pedestrians or group emerging behaviors exist. Choosing among these model generally depends on the considered crowd density or the topology of the environment. Conversely, achieving a user-desired kinematic or dynamic pattern at a given instant of the simulation reveals to be much more tedious. In this paper, a novel generic control methodology is proposed to solve this crowd editing issue. Our method relies on an adjoint formulation of the underlying optimization procedure. It is independent to a certain extent of the choice of the simulation model, and is designed to handle several forms of constraints. A variety of examples attesting the benefits of our approach are proposed, along with quantitative performance measures.     

基于智能监控的中小人群异常行为检测

针对人群异常行为检测实时性较差、分类算法识别率不高、特征量较少的问题,提出一种基于智能监控的中小人群异常行为检测算法。首先,利用快速群体密度检测算法,提取人群数量变化信息;其次,利用改进的Lucas-Kanande光流法提取视频中人群的平均动能、人群方向熵、人群距离势能;最后,利用极限学习机(ELM)算法对人群行为进行分类。使用UMN公共数据集进行测试,ELM算法对中小人群异常行为分析比中高密度人群异常行为检测算法和基于KOD能量特征的群体异常行为检测算法识别率分别高出7.13个百分点和5.89个百分点,并且人数密度估计部分平均每帧图像处理耗时相比中高密度人群异常行为检测算法减少了106 ms(近1/3)。实验结果表明:基于智能监控的中小人群异常行为检测算法能有效提高异常帧识别率和实时性。     

复杂场景下行人轨迹预测方法

为了预测行人在复杂场景中的行走轨迹,提出了一种基于生成对抗网络的可解释模型。该模型以场景中行人的历史轨迹信息和场景环境信息作为模型的输入,并在生成对抗网络中引入了物理注意力机制和社会注意力机制对行人轨迹进行预测。其中,物理注意力机制有助于建模复杂场景的整体布局并提取图像中与路径相关的显著特征,社会注意力机制能够建模不同行人之间的交互对未来轨迹的影响。在生成对抗网络的整体框架下,物理和社会注意力机制的结合使得该模型能够预测出符合物理限制和社会行为规范的多条可接受的未来路径。通过在仿真数据和真实的标准数据集上的实验,可以证明该模型能够实现对行人未来轨迹的有效预测。     

Modeling the impact of interaction on pedestrian group motion

Mobile social robots aimed at interacting with and assisting humans in pedestrian areas need to understand the dynamics of pedestrian social interaction. In this work, we investigate the effect of interaction on pedestrian group motion by defining three motion models to represent (1) interpersonal-distance, (2) relative orientation and (3) absolute difference of velocities; and model them using a dataset of 12000+ pedestrian trajectories recorded in uncontrolled settings. Our contributions include: (i) Demonstrating that interaction has a prominent effect on the empirical distributions of the proposed joint motion attributes, where increasing levels of interaction lead to more regular behavior (ii) Developing analytic motion models of such distributions and reflect the effect of interaction on model parameters (iii) Detecting the social groups in a crowd with almost perfect accuracy utilizing the proposed models, despite the constant flow direction in the environment which causes unrelated pedestrians to move in a correlated way, and thus makes group recognition more difficult (iv) Estimating the level of intensity with considerable rates utilizing the proposed models     

Shadow obstacle model for realistic corner-turning behavior in crowd simulation

This paper describes a novel model known as the shadow obstacle model to generate a realistic corner-turning behavior in crowd simulation. The motivation for this model comes from the observation that people tend to choose a safer route rather than a shorter one when turning a corner. To calculate a safer route, an optimization method is proposed to generate the corner-turning rule that maximizes the viewing range for the agents. By combining psychological and physical forces together, a full crowd simulation framework is established to provide a more realistic crowd simulation. We demonstrate that our model produces a more realistic corner-turning behavior by comparison with real data obtained from the experiments. Finally, we perform parameter analysis to show the believability of our model through a series of experiments.     

基于轨迹特征分析的行人异常行为识别

提出一个以行人为目标的视频监控系统,利用行人的移动轨迹特征来判断是否发生异常行为。利用背景相减法来检测是否有目标的存在,并利用目标的各种信息来追踪行人;建立目标行为模型,利用模型有效地识别目标的各种行为;利用轨迹对比的方式完成相关事件的轨迹检索,并供后续查询。实验表明,本算法可行、性能稳定。     

基于人机社会力模型的人群疏散算法

针对公共场合紧急情况下人群疏散困难和效果有限的问题,提出一种基于人机社会力模型的机器人疏散人群的方法。首先,基于原始社会力模型提出了一种新的人机社会力模型,该模型在原始社会力模型的基础上加入了机器人对人作用的人机作用力;然后,基于人机社会力模型提出一种新的利用机器人疏散人群的方法,该方法在人群疏散场景中加入运动机器人,通过机器人自身的运动,利用人机作用力影响周围行人的运动状态,减小行人之间的压力,从而达到加快人群运动速度、提高人群疏散效率的目的。在室内封闭场景人群逃生、两群行人交错这两种典型的疏散场景中分别进行仿真实验,并将实验结果与未加入机器人的人群疏散结果进行对比分析,实验结果表明,基于人机社会力模型的机器人疏散人群的方法能够明显加快人群的运动,提高人群的疏散效率。