Preserving interactions among moving objects in surveillance video synopsis

Video synopsis is an effective solution for fast browsing and retrieval of long surveillance videos. It aims to shorten long video sequences into its equivalent compact video representation by rearranging the video events in the temporal domain and/or spatial domain. Conventional video synopsis methods focus on reducing the collisions between tubes and maintaining their chronological order, which may alter the original interactions between tubes due to improper tube rearrangement. In this paper, we present an approach to preserve the relationships among tubes (tracks of moving objects) of the original video in the synopsis video. First, a recursive tube-grouping algorithm is proposed to determine the behavior interactions among tubes in a video and group the related tubes together to form tube sets. Second, to preserve the discovered relationships, a spatio-temporal cube voting algorithm is proposed. This cube voting method optimally rearranges the tube sets in the synopsis video, minimizing false collisions between tubes. Third, a method to estimate the duration of the synopsis video is proposed based on an entropy measure of tube collisions. The extensive experimental results demonstrate that the proposed video synopsis framework condenses videos by preserving the original tube interactions and reducing false tube collisions.

     

Teaching–learning-based optimization: A novel method for constrained mechanical design optimization problems

A new efficient optimization method, called ‘Teaching–Learning-Based Optimization (TLBO)’, is proposed in this paper for the optimization of mechanical design problems. This method works on the effect of influence of a teacher on learners. Like other nature-inspired algorithms, TLBO is also a population-based method and uses a population of solutions to proceed to the global solution. The population is considered as a group of learners or a class of learners. The process of TLBO is divided into two parts: the first part consists of the ‘Teacher Phase’ and the second part consists of the ‘Learner Phase’. ‘Teacher Phase’ means learning from the teacher and ‘Learner Phase’ means learning by the interaction between learners. The basic philosophy of the TLBO method is explained in detail. To check the effectiveness of the method it is tested on five different constrained benchmark test functions with different characteristics, four different benchmark mechanical design problems and six mechanical design optimization problems which have real world applications. The effectiveness of the TLBO method is compared with the other population-based optimization algorithms based on the best solution, average solution, convergence rate and computational effort. Results show that TLBO is more effective and efficient than the other optimization methods for the mechanical design optimization problems considered. This novel optimization method can be easily extended to other engineering design optimization problems.     

Key observation selection-based effective video synopsis for camera network

Nowadays, tremendous amount of video is captured endlessly from increased numbers of video cameras distributed around the world. Since needless information is abundant in the raw videos, making video browsing and retrieval is inefficient and time consuming. Video synopsis is an effective way to browse and index such video, by producing a short video representation, while keeping the essential activities of the original video. However, video synopsis for single camera is limited in its view scope, while understanding and monitoring overall activity for large scenarios is valuable and demanding. To solve the above issues, we propose a novel video synopsis algorithm for partially overlapping camera network. Our main contributions reside in three aspects: First, our algorithm can generate video synopsis for large scenarios, which can facilitate understanding overall activities. Second, for generating overall activity, we adopt a novel unsupervised graph matching algorithm to associate trajectories across cameras. Third, a novel multiple kernel similarity is adopted in selecting key observations for eliminating content redundancy in video synopsis. We have demonstrated the effectiveness of our approach on real surveillance videos captured by our camera network.     

Smart video summarization using mealy machine-based trajectory modelling for surveillance applications

In this paper, we propose a smart video summarization technique that compiles a synopsis of event(s)-of-interest occurring within a segment of image frames in a video. The proposed solution space consists of extracting appropriate features that represent the dynamics of targets in surveillance environments using their motion trajectories combined with a finite state automaton model for analyzing state changes of such features to detect and localize event(s)-of-interest. We introduce the cumulative moving average (CMA) and the preceding segment average (PSA) statistical metric as features that indicate gradual and sudden changes in the instantaneous velocity of moving targets. In order to support both on-line and off-line summarization, a finite state machine, that is often referred to as Mealy Machine, has been proposed to model the trajectory of a moving target and used for detecting transitions that represents a change from one state to another when initiated by a triggering event or condition. We conduct several systematic experiments on different scenario-specific in-house videos and other publicly available datasets to demonstrate the effectiveness of our proposed approach and benchmark its performance against chosen baseline strategies. The results of our experiments highlight the superiority of our proposed method in accurately localizing the start and end of event(s)-of-interest in videos within the chosen dataset.     

Probabilistic people tracking with appearance models and occlusion classification: The AD-HOC system

AD-HOC (Appearance Driven Human tracking with Occlusion Classification) is a complete framework for multiple people tracking in video surveillance applications in presence of large occlusions. The appearance-based approach allows the estimation of the pixel-wise shape of each tracked person even during the occlusion. This peculiarity can be very useful for higher level processes, such as action recognition or event detection. A first step predicts the position of all the objects in the new frame while a MAP framework provides a solution for best placement. A second step associates each candidate foreground pixel to an object according to mutual object position and color similarity. A novel definition of non-visible regions accounts for the parts of the objects that are not detected in the current frame, classifying them as dynamic, scene or apparent occlusions. Results on surveillance videos are reported, using in-house produced videos and the PETS2006 test set.     

Sparse Occlusion Detection with Optical Flow

We tackle the problem of detecting occluded regions in a video stream. Under assumptions of Lambertian reflection and static illumination, the task can be posed as a variational optimization problem, and its solution approximated using convex minimization. We describe efficient numerical schemes that reach the global optimum of the relaxed cost functional, for any number of independently moving objects, and any number of occlusion layers. We test the proposed algorithm on benchmark datasets, expanded to enable evaluation of occlusion detection performance, in addition to optical flow.     

一种基于螺旋摘要的监控视频可视分析系统

监控视频是安防系统的重要组成部分。在如今的各行各业中,只要涉及到安全,均 离不开监控视频。但对监控视频内容的分析主要依靠大量人工来完成,人力和时间成本巨大。随 着监控视频数据越来越多,如何提高针对视频内容的分析效率、降低用户认知负荷是拓展视频利 用率的重要方面。为此,针对监控视频存在的冗余信息较多、人工获取视频关键内容效率低的问 题,采用螺旋视频摘要及相应交互技术,开发了一种面向监控视频内容的可视分析系统,结合运 动目标检测结果数据,基于螺旋摘要的展示优势实现多角度可视化视频目标统计信息,并辅以针 对螺旋摘要的导航、定位操作以及草图交互等方式,实现对监控视频内容的快速有效获取。     

融合头脑风暴思想的教与学优化算法

针对教与学优化（TLBO）算法在求解高维问题时表现出的收敛速度慢、解精度低、易陷入于局部最优的问题,提出了一种融合头脑风暴思想的改进教与学优化算法（ITLBOBSO）。在该算法中设计了一种新的“学”算子,并以其替换TLBO算法中的“学”。该算法在种群的迭代过程中,当前个体首先执行“教”算子。随后,在种群中随机选择两个个体,令其中优秀的个体与当前个体执行头脑风暴式学习,提升当前个体的状态。为了赋予算法早期良好的探索能力和后期对新解的开发能力,在该算子的公式中引入柯西变异和一个与迭代次数关联的随机参数。进行的一系列的仿真实验表明,与TLBO算法相比,所提算法在11个Benchmark函数上的解精度、鲁棒性和收敛速度都有大幅度提升。在2个约束工程优化问题上,ITLBOBSO所求得的耗费成本比TLBO算法降低了4个百分点。由此验证了所提出的机制对克服TLBO弱点的有效性,所提算法适合用来求解较高维度的连续优化问题。     

Energy efficient teaching-learning-based optimization for the discrete routing problem in wireless sensor networks

Wireless sensor networks (WSNs) are composed of sensor nodes, having limited energy resources and low processing capability. Accordingly, major challenges are involved in WSNs Routing. Thus, in many use cases, routing is considered as an NP-hard optimization problem. Many routing protocols are based on metaheuristics, such as Ant Colony Optimization (ACO) and Particle Swarm Optimization (PSO). Despite the fact that metaheuristics have provided elegant solutions, they still suffer from complexity concerns and difficulty of parameter tuning. In this paper, we propose a new routing approach based on Teaching Learning Based Optimization (TLBO) which is a recent and robust method, consisting on two essential phases: Teacher and Learner. As TLBO was proposed for continuous optimization problems, this work presents the first use of TLBO for the discrete problem of WSN routing. The approach is well founded theoretically as well as detailed algorithmically. Experimental results show that our approach allows obtaining lower energy consumption which leads to a better WSN lifetime. Our method is also compared to some typical routing methods; PSO approach, advanced ACO approach, Improved Harmony based approach (IHSBEER) and Ad-hoc On-demand Distance Vector (AODV) routing protocol, to illustrate TLBO’s routing efficiency.     

多班级交互式教学优化算法

针对教学优化算法（Teaching-Learning-Based Optimization,TLBO）寻优精度低、稳定性差的问题,提出多班级交互式教学优化算法（Multi-Classes Interaction TLBO,MCITLBO）。通过引入基于欧氏距离的新型聚类划分方法,实现多班级教学,加强优秀个体周围邻域的搜索,保证算法具有较好的平衡和局部搜索能力,通过引入两种新的学习方式,实现学习方式多样化,加强种群信息交互、避免子群“滞后”或“早熟”。对6个无约束、4个约束函数和优化拉压弹簧设计问题的数值实验表明,MCITLBO相比其他算法在寻优精度和稳定性上更具优势。     

融合头脑风暴思想的教与学优化算法

针对教与学优化（TLBO）算法在求解高维问题时表现出的收敛速度慢、解精度低、易陷入于局部最优的问题，提出了一种融合头脑风暴思想的改进教与学优化算法（ITLBOBSO）。在该算法中设计了一种新的“学”算子，并以其替换TLBO算法中的“学”。该算法在种群的迭代过程中，当前个体首先执行“教”算子。随后，在种群中随机选择两个个体，令其中优秀的个体与当前个体执行头脑风暴式学习，提升当前个体的状态。为了赋予算法早期良好的探索能力和后期对新解的开发能力，在该算子的公式中引入柯西变异和一个与迭代次数关联的随机参数。进行的一系列的仿真实验表明，与TLBO算法相比，所提算法在11个Benchmark函数上的解精度、鲁棒性和收敛速度都有大幅度提升。在2个约束工程优化问题上，ITLBOBSO所求得的耗费成本比TLBO算法降低了4个百分点。由此验证了所提出的机制对克服TLBO弱点的有效性，所提算法适合用来求解较高维度的连续优化问题。     

A control benchmark on the energy management of a plug-in hybrid electric vehicle

A benchmark control problem was developed for a special session of the IFAC Workshop on Engine and Powertrain Control, Simulation and Modeling (E-COSM 12), held in Rueil-Malmaison, France, in October 2012. The online energy management of a plug-in hybrid-electric vehicle was to be developed by the benchmark participants. The simulator, provided by the benchmark organizers, implements a model of the GM Voltec powertrain. Each solution was evaluated according to several metrics, comprising of energy and fuel economy on two driving profiles unknown to the participants, acceleration and braking performance, computational performance. The nine solutions received are analyzed in terms of the control technique adopted (heuristic rule-based energy management vs. equivalent consumption minimization strategies, ECMS), battery discharge strategy (charge depleting–charge sustaining vs. blended mode), ECMS implementation (vector-based vs. map-based), ways to improve the implementation and improve the computational performance. The solution having achieved the best combined score is compared with a global optimal solution calculated offline using the Pontryagin's minimum principle-derived optimization tool HOT.     

基于聚类层次模型的视频推荐算法

目前推荐系统存在评论数据稀疏、冷启动和用户体验度低等问题,为了提高推荐系统的性能和进一步改善用户体验,提出基于聚类层次模型的视频推荐算法。首先,从相关用户方面着手,通过近邻传播（AP）聚类分析得到相似用户,从而收集相似用户中的历史网络视频数据,进而形成视频推荐集合;其次,利用用户行为的历史数据计算出用户对视频的喜好值,再把视频的喜好值转换成视频的标签权重;最后,通过层次分析模型算出视频推荐集合中用户喜好视频的排序,产生推荐列表。基于MovieLens Latest Dataset和YouTube视频评论文本数据集,实验结果表明所提算法在均方根误差和决策精度方面均表现出良好的性能。     

A video summarization approach based on the emulation of bottom-up mechanisms of visual attention

This work addresses the development of a computational model of visual attention to perform the automatic summarization of digital videos from television archives. Although the television system represents one of the most fascinating media phenomena ever created, we still observe the absence of effective solutions for content-based information retrieval from video recordings of programs produced by this media universe. This fact relates to the high complexity of the content-based video retrieval problem, which involves several challenges, among which we may highlight the usual demand on video summaries to facilitate indexing, browsing and retrieval operations. To achieve this goal, we propose a new computational visual attention model, inspired on the human visual system and based on computer vision methods (face detection, motion estimation and saliency map computation), to estimate static video abstracts, that is, collections of salient images or key frames extracted from the original videos. Experimental results with videos from the Open Video Project show that our approach represents an effective solution to the problem of automatic video summarization, producing video summaries with similar quality to the ground-truth manually created by a group of 50 users.     

Multi-objective optimal power flow using quasi-oppositional teaching learning based optimization

This paper describes teaching learning based optimization (TLBO) algorithm to solve multi-objective optimal power flow (MOOPF) problems while satisfying various operational constraints. To improve the convergence speed and quality of solution, quasi-oppositional based learning (QOBL) is incorporated in original TLBO algorithm. The proposed quasi-oppositional teaching learning based optimization (QOTLBO) approach is implemented on IEEE 30-bus system, Indian utility 62-bus system and IEEE 118-bus system to solve four different single objectives, namely fuel cost minimization, system power loss minimization and voltage stability index minimization and emission minimization; three bi-objectives optimization namely minimization of fuel cost and transmission loss; minimization of fuel cost and L-index and minimization of fuel cost and emission and one tri-objective optimization namely fuel cost, minimization of transmission losses and improvement of voltage stability simultaneously. In this article, the results obtained using the QOTLBO algorithm, is comparable with those of TLBO and other algorithms reported in the literature. The numerical results demonstrate the capabilities of the proposed approach to generate true and well-distributed Pareto optimal non-dominated solutions of the multi-objective OPF problem. The simulation results also show that the proposed approach produces better quality of the individual as well as compromising solutions than other algorithms.     

基于运动目标轨迹优化的监控视频浓缩方法

视频浓缩是包含原视频有效信息的简短表示,以便于视频的存储、浏览和检索。然而,大部分视频浓缩方法得到的浓缩视频中会丢失少量目标,不能完整表达原始视频的全部内容。本文介绍了一种基于目标轨迹优化的视频浓缩方法。首先使用改进的目标轨迹提取算法提取原视频中目标的 轨迹,然后利用马尔可夫随机场模型和松弛线性规划算法得到每条轨迹的最优时间标签,将其与背景序列和目标轨迹结合生成浓缩视频。实验结果表明,与传统的视频浓缩方法相比,本文方法生成的浓缩视频具有较高的浓缩比,保证了信息的完整性又具有良好的视觉效果。     

Requirements for multimedia metadata schemes in surveillance applications for security

Surveillance for security requires communication between systems and humans, involves behavioural and multimedia research, and demands an objective benchmarking for the performance of system components. Metadata representation schemes are extremely important to facilitate (system) interoperability and to define ground truth annotations for surveillance research and benchmarks. Surveillance places specific requirements on these metadata representation schemes. This paper offers a clear and coherent terminology, and uses this to present these requirements and to evaluate them in three ways: their fitness in breadth for surveillance design patterns, their fitness in depth for a specific surveillance scenario, and their realism on the basis of existing schemes. It is also validated that no existing metadata representation scheme fulfils all requirements. Guidelines are offered to those who wish to select or create a metadata scheme for surveillance for security.     

Illumination-robust variational optical flow using cross-correlation

We address the problem of variational optical flow for video processing applications that need fast operation and robustness to drastic variations in illumination. Recently, a solution [1] has been proposed based on the photometric invariants of the dichromatic reflection model [2]. However, this solution is only applicable to colour videos with brightness variations. Greyscale videos, or colour videos with colour illumination changes cannot be adequately handled. We propose two illumination-robust variational methods based on cross-correlation that are applicable to colour and grey-level sequences and robust to brightness and colour illumination changes. First, we present a general implicit nonlinear scheme that assumes no particular analytical form of energy functional and can accommodate different image components and data metrics, including cross-correlation. We test the nonlinear scheme on standard synthetic data with artificial brightness and colour effects added and conclude that cross-correlation is robust to both kinds of illumination changes. Then we derive a fast linearised numerical scheme for cross-correlation based variational optical flow. We test the linearised algorithm on challenging data and compare it to a number of state-of-the-art variational flow methods.     

基于自主学习行为的教与学优化算法

针对教与学优化（TLBO）算法收敛精度较低、易于早熟收敛等问题,提出一种基于自主学习行为的教与学优化算法（SLTLBO）。SLTLBO算法为学生构建了更加完善的学习框架,学生在完成常规"教"阶段与"学"阶段的学习行为之外,将进一步对比自己与教师、最差学生的差异,自主完成多样化的学习操作,以提高自己的知识水平,提高算法的收敛精度;同时学生通过高斯搜索的自主学习反思行为跳出局部区域,实现更好的全局搜索。利用10个基准测试函数对SLTLBO算法进行了性能测试,并将SLTLBO算法与粒子群优化（PSO）算法、智能蜂群（ABC）算法以及TLBO算法进行结果比对,实验结果验证了SLTLBO算法的有效性。     

SurvSurf: human retrieval on large surveillance video data

The volume of surveillance videos is increasing rapidly, where humans are the major objects of interest. Rapid human retrieval in surveillance videos is therefore desirable and applicable to a broad spectrum of applications. Existing big data processing tools that mainly target textual data cannot be applied directly for timely processing of large video data due to three main challenges: videos are more data-intensive than textual data; visual operations have higher computational complexity than textual operations; and traditional segmentation may damage video data’s continuous semantics. In this paper, we design SurvSurf, a human retrieval system on large surveillance video data that exploits characteristics of these data and big data processing tools. We propose using motion information contained in videos for video data segmentation. The basic data unit after segmentation is called M-clip. M-clips help remove redundant video contents and reduce data volumes. We use the MapReduce framework to process M-clips in parallel for human detection and appearance/motion feature extraction. We further accelerate vision algorithms by processing only sub-areas with significant motion vectors rather than entire frames. In addition, we design a distributed data store called V-BigTable to structuralize M-clips’ semantic information. V-BigTable enables efficient retrieval on a huge amount of M-clips. We implement the system on Hadoop and HBase. Experimental results show that our system outperforms basic solutions by one order of magnitude in computational time with satisfactory human retrieval accuracy.