多媒体传感器网络视频的帧分类联合重构

为降低多媒体传感器网络中视频压缩感知的计算复杂度,提出一种基于帧分类的多媒体传感器网络视频联合重构算法。依据视频数据的联合稀疏模型将视频帧分为关键帧和非关键帧。对于压缩感知重构中欠定线性方程组,可利用关键帧和非关键帧之间的相关边信息进行重构初始化,同时运用有界约束二次规划对其进行求解。从仿真结果可知,相对于传统的视频压缩感知算法而言,在保证视频重构质量的前提下,所提方法在重构算法复杂度上不但能有效降低,同时,在视频重构上能提高其实时性。     

基于内容丢帧的视频自适应传输优化策略

根据用户对视频内容的个性化偏好,提出结合语义事件与精彩度的视频内容分级方法,构建时域内基于多粒度语义内容的统一丢帧模型,在 RTP/RTSP 流媒体服务器中基于语义丢帧算法设计了视频自适应传输优化策略。实验结果表明,本文提出的基于内容的丢帧策略在网络性能、语义质量等方面优于传统基于帧类型的丢帧策略。     

基于融合特征的视频关键帧提取方法

当前对视频的分析通常是基于视频帧,但视频帧通常存在大量冗余,所以关键帧的提取至关重要.现有的传统手工提取方法通常存在漏帧,冗余帧等现象.随着深度学习的发展,相对传统手工提取方法,深度卷积网络可以大大提高对图像特征的提取能力.因此本文提出使用深度卷积网络提取视频帧深度特征与传统方法提取手工特征相结合的方法提取关键帧.首先使用卷积神经网络对视频帧进行深度特征提取,然后基于传统手工方法提取内容特征,最后融合内容特征和深度特征提取关键帧.由实验结果可得本文方法相对以往关键帧提取方法有更好的表现.     

视频解码计算复杂度的线性建模理论及在线预测方法

视频解码是一类最典型的多媒体应用,其计算量大、耗能高.现代多媒体计算平台可利用视频解码计算复杂度固有的动态变化特征来自适应地调整所需计算资源,从而节省能耗,其前提是对视频解码计算复杂度进行准确估计.作者基于解码计算复杂度与帧长之间的线性关系,提出了一种利用状态变量法对解码计算复杂度进行理论建模和在线估计的方法.与传统的直接对帧长和计算复杂度之间的输入-输出依赖关系进行建模所不同,这里将视频解码系统表征为由视频内容特征的状态变化所驱动的系统.首先从语义层面对解码器各模块的解码复杂度进行分析,并导出各模块计算复杂度与语义参数间的依赖关系模型,总解码复杂度为各子模块的复杂度之和.经过化简得到解码计算复杂度与帧长之间的线性模型,其中模型系数为上述语义参数的函数,表征了视频内容特征的状态变化,被定义为状态变量.再结合压缩视频流中相邻帧语义参数之间的相关性,将系统状态方程定义为反映视频内容变化程度的分段线性函数.根据Ⅰ帧和P帧状态轨迹特性及其在压缩码流中位置属性的不同,分别进行计算复杂度在线估计:对于I帧,采用统计分析方法获得其状态变量的均值并进行在线估计;而对P帧,则是在运行过程中利用状态方程对状态变量进行实时更新和计算复杂度估计.在基于SimpleScalar的软件仿真平台和基于DSP的嵌入式硬件平台上分别对H.264、MPEG-4压缩码流的解码计算复杂度进行在线估计,实验结果表明:对解码计算复杂度的平均估计误差在7％以内,预测精度非常高,而且状态方程更新过程简单,在线运行复杂度低,特别适用于嵌入式移动设备.     

WMSN中的分布式视频编码GOP大小控制方法

针对分布式视频编码（DVC）序列在无线多媒体传感器网络（WMSN）中传输可靠性较低的问题,提出了一种考虑节点能耗的图像组（GOP）大小控制方法。该方法分析了DVC编码框架及节点的能量消耗情况,定义了节点的可用度和帧的活动度,并对分布式视频编码序列在编码和传输过程的能耗进行了计算。在此基础上,综合考虑帧的活动度与节点能耗,实现了对分布式视频编码GOP大小的有效控制。仿真实验结果表明,与现有的GOP大小控制方法相比,该方法提高了无线多媒体传感器网络中的DVC序列的峰值信噪比,降低了节点的平均能耗,其传输可靠性也得到了显著改善。     

基于内容的视频检索综述

视频是携带信息量最大的媒体,随着抖音短视频等APP的兴起,网络以及数据库的视频数量急剧增加,人工标注的方法已经无法胜任视频检索的任务。视频检索通过提取视频帧的空间特征或者帧与帧之间的时间特征,使得用户能够更客观、更高效地进行视频查找与归类。文中概述了基于内容的视频检索算法,归纳总结了视频检索的一些经典算法,并总结了深度学习在基于内容的视频检索中的研究与应用,最后分析了深度学习在视频检索中的发展前景。     

基于DCS的WMSN多视角视频编解码

为了有效解决无线多媒体传感器网络中多视角视频监控传输数据量大以及网络能量、资源受限的问题,提出了一种基于分布式压缩感知的高压缩率多视角视频编解码方法.对多视角视频序列进行分组处理,并将图像组分为关键帧和非关键帧;对关键帧采用基于压缩感知(compressed sensing,CS)的编解码方法进行处理;而在非关键帧的编码端采用联合稀疏表示方法对残差图像稀疏表示,解码端利用帧间时间相关性和多视角空间相关性预测生成当前视频帧,并借助差异补偿方法进一步提高预测准确性,同时提高了重构效果.实验结果表明,该方法取得较高的压缩率,重构出的图像质量比参考方法更高,且PSNR值得到了较大的提高.     

基于NS-2的视频流仿真方法研究

MPEG标准将视频图像序列编码为I帧、P帧和B帧,具有压缩比高、节省网络带宽的重要特性。随着视频会议等网络多媒体应用在因特网上的部署,视频流的流量特性的仿真研究具有越来越重要的作用。文中对MPEG视频流的流量特性及其仿真方法进行了研究。在分析MPEG编码技术原理的基础上,用C／C＋＋语言扩展了NS-2仿真器内核,添加了MPEG视频流量产生器模块,对NS-2中的流量产生器模块及视频仿真接口进行了扩展。实验结果表明,所提出的NS-2视频流仿真扩展方法是可行的和有效的。     

基于不变矩和Mean Shift聚类的视频关键帧提取

随着通信和多媒体技术的迅速发展,检索和浏览海量多媒体数据成为日益迫切的问题.关键帧提取技术在基于内容的视频检索中扮演了重要角色.提取的关键帧有两个主要作用:一是用它来静态表示视频的主题;其二是希望从关键帧中提取特征数据,作为多媒体数据库的数据源.以视频场景中运动目标的不变矩为特征提出了一种基于非监督聚类的关键帧提取算法.首先在视频序列中分离出运动目标,然后计算运动目标区域的各阶不变矩,并以不变矩向量作为特征,运用Mean Shift算法聚类有相似特征的视频帧,进而在每类中选取有代表性的视频帧作为关键帧.不同场景下的实验结果证实了本算法的可行性.     

面向主观感知的视频序列帧的重要性分析

编码后的视频流经过封装形成数据包,并通过网络传输至接收端。在传输过程中视频序列的质量受到网络状态的影响,当网络出现剧烈的抖动或不稳定现象时,不可避免地会发生数据包的丢失,从而造成视频质量的损伤。利用面向主观感知的视频质量评价指标对视频序列帧的重要性进行分析,从而定义视频序列不同类型帧的重要性级别。通过实验发现,从面向主观感知的角度,P帧的重要性大于I帧,I帧的重要性又大于B帧。得到的重要性等级可以为不等差错保护以及丢帧选择提供依据。     

一种基于节的视频服务器允许接纳控制算法

提出了一种基于“服务器节”的支持压缩多媒体流的服务器中CPU、磁盘、网络和内存等资源管理的方法和允许接纳控制算法。“服务器节”概念定义了一组客户视频服务特性,如播放、快进、慢进和暂停等,并且确定了视频服务所需资源的分配参。一个“服务器节”包括视频服务器、磁盘设备、网络设备和允许接纳控制。它不但能优化使用单个资源,对于给定系统支持最大数量的客户端,保证其服务质量（QoS),而且其允许接纳控制算法能根据系统所有资源的状况,在不影响原有的视频服务基础上,确定对客户端新提出的视频服务是否接受。     

A secure framework of the surveillance video network integrating heterogeneous video formats and protocols

Recently, IT technologies are becoming focused on different traditional industries. The large scaled intelligent video surveillance system is one of them and it integrates a large number of digitalized CCTV [1] devices through the Web. However, existing video devices have been deployed for many years by different vendors as different models with different specifications. To integrate these heterogeneous devices, the centralized management server (CMS) and its clients need a specialized architecture to deal with different types of media encodings and connection protocols etc. In particular, well-defined access control mechanism is required for a large scaled surveillance video network. In this paper, we designed and implemented a server that incorporates the architecture integrating and delivering multiple video streams from different types of video devices to multiple clients and securing the access to the video streams.     

A buffered-bandwidth approach for supporting real-time video streaming over cellular networks

In this paper, we study the admission and bandwidth allocation problems in real-time video streaming in a cellular network. Admission control in a cellular network is a complex issue due to the mobility of the clients, and the additional workload imposed by incoming clients could exceed the network capacity of a cell and seriously degrade the quality of services provided to the resident clients. To minimize the number of forced terminations of real-time video playback, we incorporate the notion of buffered bandwidth in the admission test for handoff client. Using this approach, we can balance the video workload among adjacent cells to minimize the impact of overloading as the result of handoff operations. We also examine techniques to maintain fairness in services especially under overload situations even though the requested videos from various types of clients could generate very different workload. Simulation experiments confirm the effectiveness of our approach compared to widely used schemes.     

A buffered-bandwidth approach for supporting real-time video streaming over cellular networks

In this paper, we study the admission and bandwidth allocation problems in real-time video streaming in a cellular network. Admission control in a cellular network is a complex issue due to the mobility of the clients, and the additional workload imposed by incoming clients could exceed the network capacity of a cell and seriously degrade the quality of services provided to the resident clients. To minimize the number of forced terminations of real-time video playback, we incorporate the notion of buffered bandwidth in the admission test for handoff client. Using this approach, we can balance the video workload among adjacent cells to minimize the impact of overloading as the result of handoff operations. We also examine techniques to maintain fairness in services especially under overload situations even though the requested videos from various types of clients could generate very different workload. Simulation experiments confirm the effectiveness of our approach compared to widely used schemes.     

P2P视频会议系统中的媒体流组播树的生成与维护

P2P的技术把视频会议服务器的功能分散到客户端,从而解决视频会议系统中存在的网络和系统资源瓶颈问题.本文综合利用P2P技术和传统视频会议的优点,提出一种有中心管理服务器的P2P视频会议系统结构,并对节点间的数据交互和控制机制进行了研究.根据当前的Internet跨网段间不支持IP层组播的实际情况,提出在局域网内采用IP组播、在子网间采用单播的应用层组播方式进行媒体流传输.考虑P2P网络中每个节点都有自治的、动态的、自利的等特性,本文提出一种基于投标-招标机制的媒体流组播树的生成与维护算法.     

Loopback: exploiting collaborative caches for large-scale streaming

In this paper, we propose a Loopback approach in a two-level streaming architecture to exploit collaborative client/proxy buffers for improving the quality and efficiency of large-scale streaming applications. At the upper level we use a content delivery network (CDN) to deliver video from a central server to proxy servers. At the lower level a proxy server delivers video with the help of collaborative client caches. In particular, a proxy server and its clients in a local domain cache different portions of a video and form delivery loops. In each loop, a single video stream originates at the proxy, passes through a number of clients, and finally is passed back to the proxy. As a result, with limited bandwidth and storage space contributed by collaborative clients, we are able to significantly reduce the required network bandwidth, I/O bandwidth, and cache space of a proxy. Furthermore, we develop a local repair scheme to address the client failure issue for enhancing service quality and eliminating most required repairing load at the central server. For popular videos, our local repair scheme is able to handle most of single-client failures without service disruption and retransmissions from the central server. Our analysis and simulations have shown the effectiveness of the proposed scheme.     

Methodologies for designing video servers

This paper proposes an approach for integrating CPU/disk/network scheduling and memory management for supporting a variety of VCR operations and dynamic service changes efficiently. Under this approach we can optimize individual resources, and support a maximal number of clients on a given system. We present a framework of service modeling to characterize the requested video services and identify the scheduling parameters for supporting these services. A number of techniques and methodologies are developed for analyzing the behaviors of disk accesses, network operations, and CPU activities under the loads of both single and multiple clients. We also describe an admission control strategy that utilizes information about all the system resources to determine if a set of video services is acceptable     

Proxy-assisted scalable periodic broadcasting of videos for heterogeneous clients

Periodic broadcasting (PB) is a scalable technique for providing video-on-demand services. It significantly reduces server input and output (I/O) and backbone network bandwidth requirements, but increases the clients’ need for storage space and network bandwidth. Traditional protocols assume homogeneous clients with identical resources. In practice, however, clients have very different bandwidths, which are usually not sufficient for video-on-demand service from a PB server. Existing work on heterogeneous clients has focused on devising broadcast schedules that cater to low-bandwidth clients; these schedules inevitably require additional backbone network bandwidth between the server and the clients. In this paper, we propose a scheme to significantly reduce the waiting time of all heterogeneous clients, without the need for any additional backbone bandwidth. This scheme uses a proxy buffer within video-on-demand systems using PB. In the proposed system, the server broadcasts a video using one of the traditional PB protocols. Simultaneously, the proxy receives the stream from the server and stores it in its local buffer, then broadcasts the stored data to the clients in its local network. Because the proxy provides extra, transparent channels to the server, clients are likely to reduce their reception bandwidth requirements through the use of efficient reception schedules using the extra channels.     

Comparison of in-network versus Staggered Multicast video distribution models

This paper proposes a new video distribution service: Video-In-Network (VIN). In VIN, videos are continuously circulating in an optical network where they can be easily retrieved by VIN Nodes. A mathematical model of the VIN system was derived and used to explore the performance of VIN in terms of the maximum number of concurrent video streams which can be supported by the system. We then compared VIN with Staggered Multicast. We found that VIN has a number of advantages over Staggered Multicast. First, VIN is more scalable in terms of the number of streams/channels. Second, the startup latency of VIN is shorter than that of Staggered Multicast. Third, Staggered Multicast is a near-VOD system and the video clients usually have no specific channels to the server to request for video. In the VIN system, the video clients can request the VIN Node for particular videos for multicast or unicast on demand.     

基于P2P的流媒体应用及其关键算法研究

在Internet视频直播服务中,受带宽限制,服务器难以支持大规模并发客户。针对该问题,本文采用P2P方式,提高并发节点数目。文中提出网络抽象层实现P2P的网络拓扑结构,并针对对等网络中节点的不确定性、分散性以及延迟问题,详细讨论了建立和维护节点网络的BMTREE算法。最后实验表明利用BMTREE算法的P2P网络结构能高效处理视频直播流媒体应用。