Understanding Video Events: A Survey of Methods for Automatic Interpretation of Semantic Occurrences in Video

Understanding video events, i.e., the translation of low-level content in video sequences into high-level semantic concepts, is a research topic that has received much interest in recent years. Important applications of this paper include smart surveillance systems, semantic video database indexing, and interactive systems. This technology can be applied to several video domains including airport terminal, parking lot, traffic, subway stations, aerial surveillance, and sign language data. In this paper, we identify the two main components of the event understanding process: abstraction and event modeling. Abstraction is the process of molding the data into informative units to be used as input to the event model. Due to space restrictions, we will limit the discussion on the topic of abstraction. See the study by Lavee et al. (Understanding video events: A survey of methods for automatic interpretation of semantic occurrences in video, Technion—Israel Inst. Technol., Haifa, Israel, Tech. Rep. CIS-2009-06, 2009) for a more complete discussion. Event modeling is devoted to describing events of interest formally and enabling recognition of these events as they occur in the video sequence. Event modeling can be further decomposed in the categories of pattern-recognition methods, state event models, and semantic event models. In this survey, we discuss this proposed taxonomy of the literature, offer a unifying terminology, and discuss popular event modeling formalisms (e.g., hidden Markov model) and their use in video event understanding using extensive examples from the literature. Finally, we consider the application domain of video event understanding in light of the proposed taxonomy, and propose future directions for research in this field.      

A logical framework for visual information modeling and management

A unified semantic visual data-modeling framework is presented in the paper. In the proposed model, an extended conceptual graph is proposed as an annotation mechanism of a user's perceptual understanding of video objects, activities, and events. A precise definition of the term domain knowledge in visual information processing is presented. A conceptual structure, associated terms, visual feature extraction methods, and a set of constrains in feature extraction are considered as domain information. The proposed visual data model has six different abstraction layers. A higher level is more abstracted and more semantically summarized. A polygon-based bounding volume is used in video object approximation in space and time. We use a bounding volume in motion trajectory representation, rather than motion vectors. This model allows simultaneous access of both temporal and spatial information. The proposed model may be used as a referencing framework for various visual information management systems' developments.     

一种基于EDDL的非侵入式负荷检测模型

针对目前非侵入式负荷检测时存在检测精确度低的问题,提出一种基于事件驱动-深度学习(EDDL)的负荷检测模型。通过零交叉检测电流数据,基于事件驱动机制从大量数据中发现关键事件;将包含关键事件的电流序列转换至图像空间,并代入基于深度学习的负荷检测模型,从而实现端对端的非侵入式负荷检测。实验结果表明,与多分类支持向量机(MSVM)、前馈神经网络(FNN)、卷积神经网络(CNN)和长短时记忆网络(LSTM)模型相比,所提EDDL模型综合性能更优,检测准确率和精确度分别为94.67%和91.76%。仿真结果验证了所提模型可基于事件驱动机制挖掘电流数据,并基于深度学习模型有效提取电流数据特征,从而实现高精确度的非侵入式电力负荷检测。该模型对非侵入式电力负荷检测研究具有一定借鉴作用。     

基于多种词特征的微博突发事件检测方法

近年来,各领域内频频发生各类突发事件,对社会稳定发展产生了一定程度的影响.本文提出了一种基于多种词特征的微博突发事件检测模型,可以在海量微博数据中对突发事件进行检测,便于相关决策者进行微博监控和舆论引导,尽可能减少突发事件给社会带来的危害.首先根据时间信息对微博数据进行时间切片,对每一个时间窗口内的数据分别计算各个词语的词频特征、话题标签特征和词频增长率特征;然后基于D-S证据理论和层次分析法,确定词的各个特征权重,并进行加权融合得到词的突发特征值,将突发特征值大的词挑选出来构成突发特征词集,构建基于共现度和结合紧密度的突发事件特征词集的耦合度矩阵;最后将该耦合度矩阵作为凝聚式层次聚类算法的输入,生成一棵由突发词为叶子节点的二叉树,并采用内部相似度的二叉树剪枝算法对聚类结果进行划分,即可实现对相应时间窗口突发事件的检测.实验结果表明,基于突发词的事件检测模型在簇内部相似度阈值等于1.1时效果最好,正确率达到0.8462、召回率达到0.8684、F值为0.8571,表明了本文所提方法的有效性.     

Event recognition in photo albums using probabilistic graphical models and feature relevance

This paper proposes a method for event recognition in photo albums which aims at predicting the event categories of groups of photos. We propose a probabilistic graphical model (PGM) for event prediction based on high-level visual features consisting of objects and scenes, which are extracted directly from images. For better discrimination between different event categories, we develop a scheme to integrate feature relevance in our model which yields a more powerful inference when album images exhibit a large number of objects and scenes. It allows also to mitigate the influence of non-informative images usually contained in the albums. The performance of the proposed method is validated using extensive experiments on the recently-proposed PEC dataset containing over 61 000 images. Our method obtained the highest accuracy which outperforms previous work.     

基于多尺度注意力特征融合CRNN的声音事件检测

针对声音事件检测中仅在时频维度使用注意力机制的局限性以及卷积层单一导致的 特征提取不足问题,本文提出基于多尺度注意力特征融合的卷积循环神经网络(convolutional recurrent neural network,CRNN)模型,以提高声音事件检测性能。首 先,提出多尺度注意力模块,实现对局部时频单元和全局通道特征的多尺度注意,提高模型 的特征选择能力;其次,提出一种多尺度特征融合方法,融合含有丰富上下文信息的多尺度 注意力特征,提高模型的特征表达能力;最后,双向门控循环网络层对时间依赖性进行建模 , 全连接层对声音事件进行逐帧分类。除此之外,使用数据平衡技术进一步泛化模型。在 AudioSet子数据集上的实验结果表明:提出的网络模型与CRNN相比,评估集(error rate, ER)下降 11%,F1分数 (F1-score, F1)提升8.3%,有效地提高了声音事件检测性能。     

Crime hotspot prediction based on dynamic spatial analysis

Crime is not a completely random event but rather shows a pattern in space and time. Capturing the dynamic nature of crime patterns is a challenging task. Crime prediction models that rely only on neighborhood influence and demographic features might not be able to capture the dynamics of crime patterns, as demographic data collection does not occur frequently and is static. This work proposes a novel approach for crime count and hotspot prediction to capture the dynamic nature of crime patterns using taxi data along with historical crime and demographic data. The proposed approach predicts crime events in spatial units and classifies each of them into a hotspot category based on the number of crime events. Four models are proposed, which consider different covariates to select a set of independent variables. The experimental results show that the proposed combined subset model (CSM), in which static and dynamic aspects of crime are combined by employing the taxi dataset, is more accurate than the other models presented in this study.     

基于Attention深度随机森林的社区演化事件预测

在网络结构不断变化的同时,社区结构也随之演化.社区结构在不同时间片的变化可定义为四种不同的演化事件：持续、分离、融合和消失.本文运用网络表示学习的方法,对网络进行图嵌入编码映射到低维向量空间中,研究动态社区演化事件的预测.特征方面,在传统的社区内部属性特征、时间片间属性特性变化和前段时间片的社区演化事件的特征维度的基础上,引入潜在结构特征表征四种演化事件,运用随机游走和Softmax思想获取潜在的结构特征;模型方面,引入深度随机森林的策略,同时采用attention机制、蒙特卡洛特征采样策略进行特征融合和特征训练,克服了已有算法仅获取局部结构特征的缺陷.实验在DBLP、FACEBOOK和Bitcoin数据集上,对比SVM、XGBOOST和RIDGE模型训练,证实了新提出的算法模型对最终预测准确率有很大的提升.     

EEG transient event detection and classification using association rules.

In this paper, a methodology for the automated detection and classification of transient events in electroencephalographic (EEG) recordings is presented. It is based on association rule mining and classifies transient events into four categories: epileptic spikes, muscle activity, eye blinking activity, and sharp alpha activity. The methodology involves four stages: 1) transient event detection; 2) clustering of transient events and feature extraction; 3) feature discretization and feature subset selection; and 4) association rule mining and classification of transient events. The methodology is evaluated using 25 EEG recordings, and the best obtained accuracy was 87.38%. The proposed approach combines high accuracy with the ability to provide interpretation for the decisions made, since it is based on a set of association rules.     

An approximate stochastic analysis of the packet‐pair probing technique for available bandwidth estimation

The packet‐pair probing algorithm for network‐bandwidth estimation is examined and an approximate model is proposed for predicting its behaviour. The model replaces the Poisson arrival process with a Gaussian distribution and resolves the queue‐size profile into two separate components: A transient component representing the buffer‐emptying process and an equilibrium component representing the return to steady‐state behaviour. Comparison with discrete‐event simulation results shows that the model is accurate in single‐hop paths when utilization is ?70% when the cross‐traffic packets are ?½ the size of the probe packets. When extended to two‐hop paths, the model remains accurate for smaller cross‐traffic packets ($\leq\frac{1}{10}-\frac{1}{5}The packet‐pair probing algorithm for network‐bandwidth estimation is examined and an approximate model is proposed for predicting its behaviour. The model replaces the Poisson arrival process with a Gaussian distribution and resolves the queue‐size profile into two separate components: A transient component representing the buffer‐emptying process and an equilibrium component representing the return to steady‐state behaviour. Comparison with discrete‐event simulation results shows that the model is accurate in single‐hop paths when utilization is ?70% when the cross‐traffic packets are ?½ the size of the probe packets. When extended to two‐hop paths, the model remains accurate for smaller cross‐traffic packets ($\leq\frac{1}{10}-\frac{1}{5}$ the probe‐packet size). Copyright © 2008 John Wiley & Sons, Ltd.     

基于深度信念网络的事件识别

事件识别是信息抽取的重要基础.为了克服现有事件识别方法的缺陷,本文提出一种基于深度学习的事件识别模型.首先,我们通过分词系统获得候选词并将它们分为五种类型.然后选择六种识别特征并制定相应的特征表示规则用来将词转化为向量样例.最后我们通过深度信念网络抽取词的深层语义信息,并由Back-Propagation(BP)神经网络识别事件.实验显示模型最高F值达85.17%.同时,本文还提出了一种融合无监督和有监督两种学习方式的混合监督深度信念网络,该网络能够提高识别效果(F值达89.2%)并控制训练时间(增加27.50%).     

基于改进SBR算法的人脸特征点稳定检测

基于图像的特征点检测器在静态图像上取得了卓越的性能,然而这些方法应用于视频或序列图像时其精度和稳定性显著降低。配准监督(Supervision-by-Registration, SBR)算法利用光流算法(Lucas-Kanade, LK)追踪,可通过无标注视频训练针对视频的特征点检测器,已取得较好的结果,但LK算法仍存在一定局限性,导致检测的特征点序列在时空上的连贯性不强。为获得精准、稳定、连贯的人脸特征点序列检测效果,提出了平滑一致性损失函数、权重掩码函数对传统SBR网络模型进行改进。网络中添加长短期记忆网络(Long Short-Term Memory, LSTM)提高模型训练鲁棒性,在模型训练中使用平滑一致性损失函数提供稳定性约束,获得准确且稳定的人脸视频特征点检测器。在300VW、Youtube Celebrities数据集上的验证显示,SBR改进模型将人脸视频特征点检测的标准化平均误差(Normalized Mean Error, NME)从4.74降低至4.56,且视觉上人脸特征点检测的抖动显著减少。     

Unsupervised learning of parsimonious mixtures on large spaces with integrated feature and component selection

Estimating the number of components (the order) in a mixture model is often addressed using criteria such as the Bayesian information criterion (BIC) and minimum message length. However, when the feature space is very large, use of these criteria may grossly underestimate the order. Here, it is suggested that this failure is not mainly attributable to the criterion (e.g., BIC), but rather to the lack of "structure" in standard mixtures-these models trade off data fitness and model complexity only by varying the order. The authors of the present paper propose mixtures with a richer set of tradeoffs. The proposed model allows each component its own informative feature subset, with all other features explained by a common model (shared by all components). Parameter sharing greatly reduces complexity at a given order. Since the space of these parsimonious modeling solutions is vast, this space is searched in an efficient manner, integrating the component and feature selection within the generalized expectation-maximization (GEM) learning for the mixture parameters. The quality of the proposed (unsupervised) solutions is evaluated using both classification error and test set data likelihood. On text data, the proposed multinomial version-learned without labeled examples, without knowing the "true" number of topics, and without feature preprocessing-compares quite favorably with both alternative unsupervised methods and with a supervised naive Bayes classifier. A Gaussian version compares favorably with a recent method introducing "feature saliency" in mixtures.     

Activity modeling using event probability sequences.

Changes in motion properties of trajectories provide useful cues for modeling and recognizing human activities. We associate an event with significant changes that are localized in time and space, and represent activities as a sequence of such events. The localized nature of events allows for detection of subtle changes or anomalies in activities. In this paper, we present a probabilistic approach for representing events using the hidden Markov model (HMM) framework. Using trained HMMs for activities, an event probability sequence is computed for every motion trajectory in the training set. It reflects the probability of an event occurring at every time instant. Though the parameters of the trained HMMs depend on viewing direction, the event probability sequences are robust to changes in viewing direction. We describe sufficient conditions for the existence of view invariance. The usefulness of the proposed event representation is illustrated using activity recognition and anomaly detection. Experiments using the indoor University of Central Florida human action dataset, the Carnegie Mellon University Credo Intelligence, Inc., Motion Capture dataset, and the outdoor Transportation Security Administration airport tarmac surveillance dataset show encouraging results.     

显著性光流直方图字典表示的群体异常事件检测

在视频监控系统已被广泛应用的今天,基于监控视频的群体异常事件检测已成为保障社会安全的迫切需要,越来越受到人们的重视。该文基于这一现状,提出了一个新的群体异常事件检测方案,实现对监控视频自动高效的检测。在特征提取方面,提出了显著性光流直方图特征描述符,并利用该特征描述符构建字典;在字典优化方面,提出了基于聚类的多字典组合学习框架,将原始的大字典分为多个子字典;最后,对于测试样本,找出最适合的子字典并计算测试样本在该子字典下的重建误差,即可判断测试样本是否异常。在两个数据集上的实验表明,与其它方法相比,该文提出的方法对拥挤场景下监控视频中的群体异常事件检测取得了较好的检测性能。      

光纤振动信号特征提取及线性分类方法

在光纤预警系统(OFPS)中产生的入侵事件主要分 为有害入侵和无害入侵。目前对于这两类扰动常规 的特征提取方法通常是采用时域分析,但是对于不同有害入侵事件其时域特征区分不明显, 因此时域处理 不能更好体现它们之间的细节差别。通过对有害入侵信号的频谱进行统计研究发现,不同信 号的频谱分布 存在较为明显的差异性,因此本文将入侵信号变换到频域并借鉴声信号的处理方法,提出了 一种基于能量 占比特征的有害入侵事件识别算法。对采集到的振动信号进行预处理并计算功率谱密度(PSD ),计算各信号 不同频段的能量占比,并将其作为信号分类识别的特征。之后将能量占比特征作为样本送入 分类器进行 OFPS振动信号识别。在分类器的选择上,本文采用线性判别分析(LDA)分类器对信号进行识 别,LDA能 最大限度的保持原始数据信息,并有效区分振动信号。通过实验结果表明该算法在OFPS 振 动信号的识别 研究中提高了有害入侵信号的识别率,从而验证了本算法的可行性,同时有效减少了识别时 间。     

纳米级CMOS集成电路的单粒子效应及其加固技术

空间应用的集成电路受到辐射效应的影响,会出现瞬态干扰、数据翻转、性能退化、功能失效甚至彻底毁坏等问题.随着器件特征尺寸进入到100nm以下（以下简称纳米级）,这些问题的多样性和复杂性进一步增加,单粒子效应成为集成电路在空间可靠性应用的主要问题,给集成电路的辐射效应评估和抗辐射加固带来了诸多挑战.本文以纳米级CMOS集成电路为研究对象,结合近年来国内外的主要技术进展,介绍研究团队在65nm集成电路单粒子效应和加固技术方面的研究成果,包括首次提出的单粒子时域测试和分析方法、单粒子多节点翻转加固方法和单粒子瞬态加固方法等.     

基于VMD与MFE的光纤周界安防入侵事件识别北大核心CSCD

为了提高周界入侵事件的识别率,本文提出一种基于超弱光纤光栅(UWFBG)阵列的光纤周界入侵事件识别方法。该方法通过变分模态分解(VMD)将入侵信号进行分解,然后选择最佳分量并提取其多尺度模糊熵(MFE),与信号过零率(ZCR)相结合构造特征向量,将其输入到Sigmoid函数拟合的支持向量机(SVM),实现对晃动、剪切、刮风、下雨和无入侵5种不同的事件的识别。实验表明,该方法可以准确识别5种常见的事件信号,平均识别率达到98%。此外,该方法可以在输出各入侵事件类别的同时输出各类事件发生的概率值。