QoE management for video conferencing applications

We introduce a framework for managing the QoE of videos coded with the H.264 codec and transmitted by video conferencing applications through limited bandwidth networks. We focus our study on the medium-motion videos with QCIF, CIF, and VGA resolutions, the most pervasive video formats used by video conferencing applications across the Internet and cellular telephony systems. Using subjective tests for measuring the level of video quality perceived by end users, we expose the relation between the main influential video parameters and the quality experienced by end users. Furthermore, after investigating the effect of different frame rates and compression levels on video streaming bit rate, and consequently on QoE, we propose a QoE control mechanism for limited-bandwidth situations. A congestion control technique is also introduced in this paper and used in simulations for verifying the efficiency of the proposed QoE management algorithm and to implement this algorithm for practical applications.     

Impact of video content and transmission impairments on quality of experience

The analysis of the impact of video content and transmission impairments on Quality of Experience (QoE) is a relevant topic for the robust design and adaptation of multimedia infrastructures, services, and applications. The goal of this paper is to study the impact of video content on QoE for different levels of impairments. In more details, this contribution aims at i) the study of the impact of delay, jitter, packet loss, and bandwidth on QoE, ii) the analysis of the impact of video content on QoE, and iii) the evaluation of the relationship between content related parameters (spatial-temporal perceptual information, motion, and data rate) and the QoE for different levels of impairments.     

智能视觉监控技术研究进展

新一代智能视觉监控技术的研究是一个极具挑战性的前沿课题,它旨在赋予监控系统观察分析场景内容的能力,实现监控的自动化和智能化,因而具有巨大的应用潜力。视觉监控系统的智能化分析过程由运动目标检测、分类、跟踪和视频内容分析等几个基本环节组成,其中视频内容分析又包括异常检测、人的身份识别以及视频内容理解描述等。本文在总结以上有关关键技术研究进展的基础上,进一步提出将超分辨率复原技术引入视觉监控领域,介绍了超分辨率复原的主要算法及其在智能视觉监控中的应用。     

基于半监督特征融合的监控视频场景识别研究

针对单模态特征条件下监控视频的场景识别精度与鲁棒性不高的问题,提出一种基于特征融合的半监督学习场景识别系统.系统模型首先通过卷积神经网络预训练模型分别提取视频帧与音频的场景描述特征;然后针对场景识别的特点进行视频级特征融合;接着通过深度信念网络进行无监督训练,并通过加入相对熵正则化项代价函数进行有监督调优;最后对模型分...     

人群密集度下GAN的视频异常行为检测进展

视频异常检测作为计算机视觉的重要分支,是智能监控系统中一项极具挑战性的任务,通常是指自动识别视频中的异常目标、行为或事件,对保障公共安全起着至关重要的作用。生成对抗网络是一种新兴的无监督方法,不仅可以用于生成图像,且其独特的对抗性学习思想在异常检测领域也显示出良好的发展潜力。介绍了生成对抗网络的框架结构;根据场景密度以及行为发生的对象,从个体行为异常、群体异常两个方面论述了生成对抗网络在视频异常检测领域的研究现状,分别基于重构和预测的方法对个体异常行为检测和群体异常行为检测作进一步阐述;简要介绍了视频异常检测的常用数据集;最后对未来发展作出了展望。     

基于Spark的实时视频分析系统

视频监控技术在交通管理、公共安全、智慧城市等方面有着广泛的应用前景,且向着智能识别、实时处理、大数据分析的方向发展. 本文针对大规模实时视频监控提出了新的解决方案. 基于Spark streaming流式计算、分布式存储及OLAP框架,使多路视频处理在可扩展性、容错性及数据多维聚合分析上具有明显的优势. 系统根据视频处理算法划分为单机处理与分布式处理. 并将视频图像处理与数据分析耦合,利用Kafka消息队列与Spark streaming完成对多路视频输出数据的进一步操作. 结合分布式存储方案,并利用OLAP框架实现对海量数据实时多维聚合分析与高效实时查询.     

Multipoint-to-point communications for SHE surveillance with QoS and QoE management

The bandwidth-hunger applications of SHE (Smart Home Environment) can take advantage of the multipoint-to-point (MPP) connections to aggregate more bandwidth to gain user-perceived Quality of Experience (QoE) and network Quality of Service (QoS). The receiver-centric transport-layer R²CP (Radial Reception Control Protocol) was proposed to resolve the incapability of the MPP communication in conventional TCP and UDP. However, R²CP has no consideration to discriminate the importance in a packet payload which is critical to QoE and brings an issue for critical data packets that may be dropped in great risk of network congestion. In this paper, we thus present P-R²CP (Prioritized R²CP) to effectively decrease the loss ratio of critical data packets in MPP video streaming while the network is congested. P-R²CP is a cross-layer protocol that considers both the transport-layer issues and the media content’s properties in application-layer. Then, an example on MPP-UVS (MPP ubiquitous video surveillance) is presented as UVS is now a very important Internet application that requires QoS/QoE management to protect lives and assets especially in SHE. Our experiments are conducted on different kinds of surveillance videos over MPP links with different bandwidth and packet loss inserted. The experimental results demonstrate that, as the loss of critical packets is decreased by an order and much less critical data packets are dropped, P-R²CP can highly guard not only QoS but also QoE of SHE surveillance video streaming.     

从传统到深度：视觉烟雾识别、检测与分割

在烟雾检测系统中,采用机器学习的视觉技术暂未广泛替代传感器的主要原因在于其误报与漏报较高。计算力度的提高、存储设备的发展,使得传统视觉技术中存在的问题逐渐得到改善或解决,但也迎来了新的挑战。为反映用于森林火灾预警的烟雾识别、检测等技术的最新研究进展,本文重点对2017—2019年国内外公开发表的相关文献进行梳理和分析。从监控角度出发,基于对此领域的长期研究与广泛文献调研,将利用烟雾的森林火灾预警任务分为烟雾识别、检测、分割这3类不同的粒度,分别介绍实现这些任务的传统方法及深度方法。依照当前研究热度,主要关注视频烟雾检测与分割这两个细粒度任务。其中烟雾区域的粗提取与二次提取方法是检测与分割的关键,因此将探索这些方法如何提取、利用烟雾的动态与静态特征。此外,由于深度学习框架主要实现端对端的任务,无法分离出关键步骤,故对基于深度学习的烟雾监控任务进行单独梳理,不关注单步细节,主要体现文献思路。最后,对实现烟雾识别、检测、分割任务具体方法中的优缺点、烟雾监控任务中常用的指标、研究常用的数据库进行总结,并对发展前景进行展望。为基于烟雾的森林火灾预警技术提供更多的发展方向。     

Real‐time smart surveillance using motion analysis

Abstract: In the last years, smart surveillance has been one of the most active research topics in computer vision because of the wide spectrum of promising applications. Its main point is about the use of automatic video analysis technologies for surveillance purposes. In general, a processing framework for smart surveillance consists of a preliminary motion detection step in combination with high‐level reasoning that allows automatic understanding of evolutions of observed scenes. In this paper, we propose a surveillance framework based on a set of reliable visual algorithms that perform different tasks: a motion analysis approach that segments foreground regions is followed by three procedures, which perform object tracking, homographic transformations and edge matching, in order to achieve the real‐time monitoring of forbidden areas and the detection of abandoned or removed objects. Several experiments have been performed on different real image sequences acquired from a Messapic museum (indoor context) and the nearby archaeological site (outdoor context) to demonstrate the effectiveness and the flexibility of the proposed approach.     

Big data analytics for video surveillance

This article addresses the usage and scope of Big Data Analytics in video surveillance and its potential application areas. The current age of technology provides the users, ample opportunity to generate data at every instant of time. Thus in general, a tremendous amount of data is generated every instant throughout the world. Among them, amount of video data generated is having a major share. Education, healthcare, tours and travels, food and culture, geographical exploration, agriculture, safety and security, entertainment etc., are the key areas where a tremendous amount of video data is generated every day. A major share among it are taken by the daily used surveillance data captured from the security purpose camera and are recorded everyday. Storage, retrieval, processing, and analysis of such gigantic data require some specific platform. Big Data Analytics is such a platform, which eases this analysis task. The aim of this article is to investigate the current trends in video surveillance and its applications using Big Data Analytics. It also aims to focus on the research opportunities for visual surveillance in Big Data frameworks. We have reported here the state-of-the-art surveillance schemes for four different imaging modalities: conventional video scene, remotely sensed video, medical diagnostics, and underwater surveillance. Several works were reported in this research field over recent years and are categorized based on the challenges solved by the researchers. A list of tools used for video surveillance using Big Data framework is presented. Finally, research gaps in this domain are discussed.

     

基于Spark Streaming的视频/图像流处理与新的性能评估方法

智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的数据处理问题提供了很好的途径。使用一种基于Spark Streaming的视频/图像流处理的测试平台,阐述了平台的构成和工作流程,深入研究各个参数对集群性能的影响,创新性地提出了CPU时间占用率作为性能评估指标,与总的处理时间结合,更为全面反映集群性能和资源利用率。     

一种智能视频监控体系结构设计方案

智能视频监控就是将自动视频分析技术应用到视频监控系统中。本文探讨了智能视频监控的功能作用,设计了一种智能视频监控系统的体系结构方案,并重点阐述了其智能监控服务器模块的内部功能和结构,指出了实现智能监控所需的关键技术。     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

A comprehensive survey of human action recognition with spatio-temporal interest point (STIP) detector

Over the past two decades, human action recognition from video has been an important area of research in computer vision. Its applications include surveillance systems, human–computer interactions and various real-world applications where one of the actor is a human being. A number of review works have been done by several researchers in the context of human action recognition. However, it is found that there is a gap in literature when it comes to methodologies of STIP-based detector for human action recognition. This paper presents a comprehensive review on STIP-based methods for human action recognition. STIP-based detectors are robust in detecting interest points from video in spatio-temporal domain. This paper also summarizes related public datasets useful for comparing performances of various techniques.     

Shot Partitioning Based Recognition of TV Commercials

Digital video applications exploit the intrinsic structure of video sequences. In order to obtain and represent this structure for video annotation and indexing tasks, the main initial step is automatic shot partitioning. This paper analyzes the problem of automatic TV commercials recognition, and a new algorithm for scene break detection is then introduced. The structure of each commercial is represented by the set of its key-frames, which are automatically extracted from the video stream. The particular characteristics of commercials make commonly used shot boundary detection techniques obtain worse results than with other video content domains. These techniques are based on individual image features or visual cues, which show significant performance lacks when they are applied to complex video content domains like commercials. We present a new scene break detection algorithm based on the combined analysis of edge and color features. Local motion estimation is applied to each edge in a frame, and the continuity of the color around them is then checked in the following frame. By separately considering both sides of each edge, we rely on the continuous presence of the objects and/or the background of the scene during each shot. Experimental results show that this approach outperforms single feature algorithms in terms of precision and recall.     

User-centric privacy awareness in video surveillance

Especially in urban environments, video cameras have become omnipresent. Supporters of video surveillance argue that it is an excellent tool for many applications including crime prevention and law enforcement. While this is certainly true, it must be questioned if sufficient efforts are made to protect the privacy of monitored people. Privacy concerns are often set aside when compared to public safety and security. One reaction to this situation is emerging: community-based efforts where citizens register and map surveillance cameras in their environment. Our study is inspired by this idea and proposes a user-specific and location-aware privacy awareness system. Using conventional smartphones, users not only can contribute to the camera maps, but also use community-collected data to be alerted of potential privacy violations. In our model, we define different levels of privacy awareness. For the highest level, we present a mechanism that allows users to directly interact with specially designed, trustworthy cameras. These cameras provide direct feedback about the tasks that are executed by the camera and how privacy-sensitive data is handled. A hardware security chip that is integrated into the camera is used to ensure authenticity, integrity and freshness of the provided camera status information.     

Selective deep ensemble for instance retrieval

In public security systems, visual instance retrieval has an explosive growing requirement, especially for large-scale image or video databases. Due to its wide range of applications in surveillance scenario, this paper aims at the retrieval tasks centered around ‘vehicle’ and ‘pedestrian’ targets. Many previous CNN-based methods have not exploited the ensemble abilities of different models, which achieve limited accuracy since a certain kind of deep architecture is not comprehensive. On the other hand, some features in the original deep representation are useless for retrieval tasks, while the attention-aware compact representation will be much more efficient and effective. To address the above problems, we propose a Selective Deep Ensemble (SDE) framework to combine various models and features in a complementary way, inspired by the attention mechanism. It is demonstrated that a large improvement can be acquired with slight increase on computation cost. Finally, we evaluate the performance on three public instance-retrieval datasets, VehicleID, VeRi and Market-1501, outperforming state-of-the-art methods by a large margin.