HOPE: a hybrid deep neural model for out-of-town next POI recommendation

Next Point-of-interest (POI) recommendation has been recognized as an important technique in location-based services, and existing methods aim to utilize sequential models to return meaningful recommendation results. But these models fail to fully consider the phenomenon of user interest drift, i.e. a user tends to have different preferences when she is in out-of-town areas, resulting in sub-optimal results accordingly. To achieve more accurate next POI recommendation for out-of-town users, an adaptive attentional deep neural model HOPE is proposed in this paper for modeling user’s out-of-town dynamic preferences precisely. Aside from hometown preferences of a user, it captures the long and short-term preferences of the user in out-of-town areas using “Asymmetric-SVD” and “TC-SeqRec” respectively. In addition, toward the data sparsity problem of out-of-town preference modeling, a region-based pattern discovery method is further adopted to capture all visitor’s crowd preferences of this area, enabling out-of-town preferences of cold start users to be captured reasonably. In addition, we adaptively fuse all above factors according to the contextual information by adaptive attention, which incorporates temporal gating to balance the importance of the long-term and short-term preferences in a reasonable and explainable way. At last, we evaluate the HOPE with baseline sequential models for POI recommendation on two real datasets, and the results demonstrate that our proposed solution outperforms the state-of-art models significantly.

     

一种抗背景干扰的多尺度人群计数算法

人群计数技术以估计人群图片或视频中的人数为目标,可以有效预防人群踩踏事故的发生,广泛应用于安防预警、城市规划及大型集会管理等领域。然而,由于人群尺度变化、背景干扰、人群分布不均、遮挡和透视效应等因素的影响,单幅图片的人群计数仍是一项非常具有挑战性的任务。针对人群计数中多尺度变化和背景干扰问题,提出一种抗背景干扰的多尺度人群计数算法。以VGG16网络结构为基础,引入特征金字塔构建多尺度特征融合骨干网络解决人群多尺度变化问题,设计Double-Head-CC结构对融合后的特征图进行前景背景分割和密度图预测以抑制背景干扰。基于密度图的局部相关性和多任务学习,定义多重损失函数和多任务联合损失函数进行网络优化。在ShanghaiTech、UCF-QNRF和JHU-CROWD++数据集上进行训练和评测,实验结果表明,该算法能够很好地预测人群密度分布和人群数量,具有较高的准确性,且鲁棒性强、泛化性能良好。     

People counting via multiple views using a fast information fusion approach

Real-time estimates of a crowd size is a central task in civilian surveillance. In this paper we present a novel system counting people in a crowd scene with overlapping cameras. This system fuses all single view foreground information to localize each person present on the scene. The purpose of our fusion strategy is to use the foreground pixels of each single views to improve real-time objects association between each camera of the network. The foreground pixels are obtained by using an algorithm based on codebook. In this work, we aggregate the resulting silhouettes over cameras network, and compute a planar homography projection of each camera’s visual hull into ground plane. The visual hull is obtained by finding the convex hull of the foreground pixels. After the projection into the ground plane, we fuse the obtained polygons by using the geometric properties of the scene and on the quality of each camera detection. We also suggest a region-based approach tracking strategy which keeps track of people movements and of their identities along time, also enabling tolerance to occasional misdetections. This tracking strategy is implemented on the result of the views fusion and allows to estimate the crowd size dependently on each frame. Assessment of experiments using public datasets proposed for the evaluation of counting people system demonstrates the performance of our fusion approach. These results prove that the fusion strategy can run in real-time and is efficient for making data association. We also prove that the combination of our fusion approach and the proposed tracking improve the people counting.

     

基于单列多尺度卷积神经网络的人群计数

单张图片和监控视频中的人群计数问题在近年来受到了越来越多的关注。尺度的变化和人群遮挡等问题,导致人群计数是一项十分具有挑战性的任务,但是深度卷积神经网络被证明能有效地解决这一问题。文中提出了一种单列多尺度的卷积神经网络,该网络提供了一种数据驱动的深度学习方法,能够理解各种不同的场景,并能进行精确的计数估计。该网络模型主要由作为二维特征提取的前端与中端,和用来还原密度图的后端组成。其中,使用堆叠池代替最大池化层,在不引入额外参数的前提下增加了模型的尺度不变性。网络模型前端采用部分VGG-16结构;中端采用FME(特征聚合模块),用来打破不同列之间的独立,以更好地提取多尺度特征信息;后端采用3列5层的不同扩张率的空洞卷积,在保持分辨率不变的情况下增加感受野,生成更高质量的人群密度图,并引入一种相对人数损失,以提升稀疏密度人群情况下模型的性能。该模型在两个最具挑战性的人群计数数据集上都取得了很好的效果。实验结果表明,在公开人群计数数据集ShanghaiTech的两个子集和UCF_CC_50上,该方法的平均绝对误差(MAE)和均方误差(MSE)分别是66.2和103.0、8.7和13.4、251.0和329.5,性能比传统人群计数方法更好。与其他模型相比,该模型拥有更高的精度和更好的鲁棒性,对稀疏人数图像有着更好的计数效果。     

Adaptive Scheme for Crowd Counting Using off-the-Shelf Wireless Routers

Since the outbreak of the world-wide novel coronavirus pandemic, crowd counting in public areas, such as in shopping centers and in commercial streets, has gained popularity among public health administrations for preventing the crowds from gathering. In this paper, we propose a novel adaptive method for crowd counting based on Wi-Fi channel state information (CSI) by using common commercial wireless routers. Compared with previous researches on device-free crowd counting, our proposed method is more adaptive to the change of environment and can achieve high accuracy of crowd count estimation. Because the distance between access point (AP) and monitor point (MP) is typically non-fixed in real-world applications, the strength of received signals varies and makes the traditional amplitude-related models to perform poorly in different environments. In order to achieve adaptivity of the crowd count estimation model, we used convolutional neural network (ConvNet) to extract features from correlation coefficient matrix of subcarriers which are insensitive to the change of received signal strength. We conducted experiments in university classroom settings and our model achieved an overall accuracy of 97.79% in estimating a variable number of participants.     

Approaches on crowd counting and density estimation: a review

In recent years, urgent needs for counting crowds and vehicles have greatly promoted research of crowd counting and density estimation. Benefiting from the rapid development of deep learning, the counting performance has been greatly improved, and the application scenarios have been further expanded. Aiming to deeply understand the development status of crowd counting and density estimation, we introduce and analyze the typical methods in this field and especially focus on elaborating deep learning-based counting methods. We summarize the existing approaches into four categories, i.e., detection-based, regression-based, convolutional neural network based and video-based. Each category is explicated in great detail. To provide more concrete reference, we compare the performance of typical methods on the popular benchmarks. We further elaborate on the datasets and metrics for the crowd counting community and discuss the work of solving the problem of small-sample-based counting, dataset annotation methods and so on. Finally, we summarize various challenges facing crowd counting and their corresponding solutions and propose a set of development trends in the future.

     

基于多层BP神经网络和无参数微调的人群计数方法

针对大部分现有的人群计数方法被应用到新的场景时性能下降的问题,在多层BP神经网络框架下,提出一种具有无参数微调的人群计数方法。首先,从训练图像中裁切图像块,将获得的相似尺度的行人作为人群BP神经网络模型的输入;然后,BP神经网络模型通过学习预测密度图,得到了一个具有代表性的人群块;最后,为了处理新场景,对训练好的BP神经网络模型进行目标场景微调,可追求有相同属性的样本,包括候选块检索和局部块检索。实验数据集包括PETS2009数据集、UCSD数据集和UCF_CC_50数据集。这些场景的实验结果验证了提出方法的有效性。相比于全局回归计数法和密度估计计数法,提出的方法在平均绝对误差和均方误差方面均有较大优势, 消除了场景间区别和前景分割的影响。     

Learning behavior patterns from video for agent-based crowd modeling and simulation

This paper proposes a novel data-driven modeling framework to construct agent-based crowd model based on real-world video data. The constructed crowd model can generate crowd behaviors that match those observed in the video and can be used to predict trajectories of pedestrians in the same scenario. In the proposed framework, a dual-layer architecture is proposed to model crowd behaviors. The bottom layer models the microscopic collision avoidance behaviors, while the top layer models the macroscopic crowd behaviors such as the goal selection patterns and the path navigation patterns. An automatic learning algorithm is proposed to learn behavior patterns from video data. The learned behavior patterns are then integrated into the dual-layer architecture to generate realistic crowd behaviors. To validate its effectiveness, the proposed framework is applied to two different real world scenarios. The simulation results demonstrate that the proposed framework can generate crowd behaviors similar to those observed in the videos in terms of crowd density distribution. In addition, the proposed framework can also offer promising performance on predicting the trajectories of pedestrians.     

Directional pedestrian counting with a hybrid map-based model

Detection-based pedestrian counting methods produce results of considerable accuracy in non-crowded scenes. However, the detection-based approach is dependent on the camera viewpoint. On the other hand, map-based pedestrian counting methods are performed by measuring features that do not require separate detection of each pedestrian in the scene. Thus, these methods are more effective especially in high crowd density. In this paper, we propose a hybrid map-based model that is a new directional pedestrian counting model. Our proposed model is composed of direction estimation module with classified foreground motion vectors, and pedestrian counting module with principal component analysis. Our contributions in this paper have two aspects. First, we present a directional moving pedestrian counting system that does not depend on object detection or tracking. Second, the number and major directions of pedestrian movements can be detected, by classifying foreground motion vectors. This representation is more powerful than simple features in terms of handling noise, and can count the moving pedestrians in images more accurately.

     

基于密度分类及组合特征的人数估计算法*

为了克服不同人群密度及所采用特征对人数估计的影响,提出了一种基于人群密度分类及组合特征的人数统计算法。该算法包括离线特征组合选取和在线实时估计两个阶段。在离线阶段,选取密度阈值将图像样本分为高、低密度两类,然后通过实验方法选取最优的特征组合。在线估计阶段首先通过分类器将样本分为高、低密度两类,然后利用离线阶段选取的特征组合训练得到高斯模型,并分别对两类样本进行人数估计。实验结果表明,与不分高低密度相比,平均估计误差由10.6%降至8.1%;与目前主流的人数估计算法相比,本文算法的平均估计误差也更小。     

基于对抗式扩张卷积的多尺度人群密度估计

目的 人群密度估计任务是通过对人群特征的提取和分析,估算出密度分布情况和人群计数结果。现有技术运用的CNN网络中的下采样操作会丢失部分人群信息,且平均融合方式会使多尺度效应平均化,该策略并不一定能得到准确的估计结果。为了解决上述问题,提出一种新的基于对抗式扩张卷积的多尺度人群密度估计模型。方法 利用扩张卷积在不损失分辨率的情况下对输入图像进行特征提取,且不同的扩张系数可以聚集多尺度上下文信息。最后通过对抗式损失函数将网络中提取的不同尺度的特征信息以合作式的方式融合,得到准确的密度估计结果。结果 在4个主要的人群计数数据集上进行对比实验。在测试阶段,将测试图像输入训练好的生成器网络,输出预测密度图;将密度图积分求和得到总人数,并以平均绝对误差（MAE）和均方误差（MSE）作为评价指标进行结果对比。其中,在ShanghaiTech数据集上Part_A的MAE和MSE分别降至60.5和109.7,Part_B的MAE和MSE分别降至10.2和15.3,提升效果明显。结论 本文提出了一种新的基于对抗式扩张卷积的多尺度人群密度估计模型。实验结果表明,在人群分布差异较大的场景中构建的算法模型有较好的自适应性,能根据不同的场景提取特征估算密度分布,并对人群进行准确计数。     

A Hybrid Model for Simulating Crowd Evacuation

Macroscopic and microscopic models are typical approaches for simulating crowd behaviour and movement to simulate crowd and pedestrian movement, respectively. However, the two models are unlikely to address the issues beyond their modelling targets (i.e., pedestrian movement for microscopic models and crowd movement for macroscopic models). In order to solve such problem, we propose a hybrid model integrating macroscopic model into microscopic model, which is capable of taking into account issues both from crowd movement tendency and individual diversity to simulate crowd evacuation. In each simulation time step, the macroscopic model is executed first and generates a course-grain simulation result depicting the crowd movement, which directs microscopic model for goal selection and path planning to generate a fine-grain simulation result. In the mean time, different level-of-detail simulation results can also be obtained due to the proposed model containing two complete models. A synchronization mechanism is proposed to convey simulation results from one model to the other one. The simulation results via case study indicate the proposed model can simulate the crowd and agent behaviour in dynamic environments, and the simulation cost is proved to be efficient.     

基于像素级注意力机制的人群计数方法

针对人群分布不均和网络学习参数众多问题,提出了一种由像素级注意力机制（PAM）和改进的单列人群密度估计网络两部分组成的高密度人群计数方法。首先,使用PAM通过对人群图像进行像素级别的分类来生成高质量的局部人群密度图,利用全卷积网络（FCN）生成每个图像的密度掩码,将图像中的像素分为不同的密度级别;然后,以生成的密度掩码为标签,使用单列人群密度估计网络以更少的参数学习到更多的代表性特征。在此之前,在Shanghaitech数据集part_B部分、UCF_CC_50数据集以及WorldExpo'10数据集上,拥塞场景识别网络（CSRNet）方法的计数误差最小。将所提方法与CSRNet方法的误差结果对比,发现所提方法在Shanghaitech数据集part_B部分的平均绝对误差（MAE）和均方误差（MSE）分别降低了8.49%和4.37%;在UCF_CC_50数据集上的MAE和MSE分别降低了58.38%和51.98%,优化效果显著;在WorldExpo'10数据集上的整体平均值部分的MAE降低了1.16%。实验结果表明,在针对人群分布不均的高密度人群计数时,结合PAM和单列人群密度估计网络的方法能够有效提高高密度人群计数的精确度和训练效率。     

结合最短路径改进的社会力人群疏散仿真模型

社会力模型广泛应用于人群疏散仿真,针对该模型在仿真过程中存在行人停滞不前、无法通过非凸边形障碍物和疏散路径与行人实际选择的路径不相符等问题,提出了一种社会力改进模型。该模型基于场景中的障碍物生成路径节点,利用这些节点生成无向图,同时考虑了节点的安全系数和拥挤系数对节点通行性的影响生成最短疏散路径。通过改进后的社会力模型进行了多种场景的仿真实验,实验结果显示行人在复杂障碍物场景中能有效绕过障碍物,生成合理的疏散路径,表明该模型有效改善社会力模型,使人群疏散仿真更加真实。     

Pyramid-dilated deep convolutional neural network for crowd counting

Statistics on crowds in crowded scenes can reflect the density level of crowds and provide safety warnings. This is a laborious task if conducted manually. In recent years, automated crowd counting has received extensive attention in the computer vision field. However, this task is still challenging mainly due to the serious occlusion in crowds and large appearance variations caused by the viewing angles of cameras. To overcome these difficulties, a pyramid-dilated deep convolutional neural network for accurate crowd counting called PDD-CNN is proposed. PDD-CNN is based on a VGG-16 network that is designed to generate dense attribute feature maps from an image with an arbitrary size or resolution. Then, two pyramid dilated modules are adopted, each consisting of four parallel dilated convolutional layers with different rates and a parallel average pooling layer to capture the multiscale features. Finally, three cascading dilated convolutions are used to regress the density map and perform accurate count estimation. In addition, a novel training loss, combining the Euclidean loss with the structural similarity loss, is employed to attenuate the blurry effects of density map estimation. The experimental results on three datasets (ShanghaiTech, UCF_CC_50, and UCF-QNRF) demonstrate that the proposed PDD-CNN produces high-quality density maps and achieves a good counting performance.

     

基于多尺度卷积神经网络的人群聚集异常预测

已有的公共场所人群聚集异常行为检测方法较少,且大多检测方法都是在人群已经异常聚集后再进行检测,检测准确率不高,时效性不够好。提出一种基于多尺度卷积神经网络（MCNN）的人群聚集异常预测模型。首先,通过多尺度卷积神经网络训练一个人群计数模型,用训练好的模型对人群聚集异常视频进行测试;然后在测试中完成人群人数统计与人群头部坐标点获取,进而计算人群密度、人群距离势能与人群分布熵;最后将得到的3种人群运动状态特征值利用PSO-ELM进行训练,得到预测模型,通过特征数据的变化,完成人群聚集行为的预测。实验结果表明,与现有算法相比,该模型能有效实现人群聚集异常行为的预警与检测,时效性强,为采取相应应急措施提供了更多时间,预测准确率达到了9717%。     

COMAL: compositional multi-scale feature enhanced learning for crowd counting

Multimedia Tools and Applications - Accurately modeling the crowd’s head scale variations is an effective way to improve the counting accuracy of the crowd counting methods. Most counting...     

基于多尺度多任务卷积神经网络的人群计数

在智能监控领域，实现人群计数具有重要价值，针对人群尺度不一、人群密度分布不均及遮挡等问题，提出一种多尺度多任务卷积神经网络（MMCNN）进行人群计数的方法。首先提出一种新颖的自适应人形核生成密度图描述人群信息，消除人群遮挡影响；其次通过构建多尺度卷积神经网络解决人群尺度不一问题，以多任务学习机制同时估计密度图及人群密度等级，解决人群分布不均问题；最后设计一种加权损失函数，提高人群计数准确率。在UCF_CC_50和World Expo'10数据库上进行了评估，验证了自适应人形核的有效性。实验结果表明：所提算法比Sindagi等的方法（SINDAGI V A，PATEL V M.CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting.Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance.Piscataway，NJ：IEEE，2017：1-6）在UCF_CC_50数据库上平均绝对误差（MAE）数值和均方误差（MSE）数值分别降低约1.7和45；与Zhang等的方法（ZHANG Y，ZHOU D，CHEN S，et al.Single-image crowd counting via multi-column convolutional neural network.Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington，DC：IEEE Computer Society，2016：589-597）相比，在World Expo'10数据库上所提算法的MAE值降低约1.5，且在真实公共汽车数据库上仅0~3人的计数误差，表明其实用性较强。