Correlation-based and content-enhanced network for video style transfer

Pattern Analysis and Applications - Artistic style transfer aims to migrate the style pattern from a referenced style image to a given content image, which has achieved significant advances in...     

适于实时传输的AVS视频码流加密算法*

为应对日益严重的盗版问题,提出了一种针对AVS标准的视频信息加密算法。通过加密运动矢量,使得视频图像重建时无法找到匹配的参考块,并通过误差扩散,引起视频图像错乱,从而达到对视频信息加密的目的。加密算法不改变码流结构,因此,接收端未经解密的视频码流可以由标准解码器正常解码,但是解码重建的视频图像质量明显下降,而经解密的视频重建图像可完全恢复到加密前的水平。实验结果表明,提出的加密方案具有良好的加密效果,其算法复杂度低,对编码效率影响也很小,基本上不影响视频的主客观质量。     

大数据量实时视频流单播传输拥塞控制机制研究

通过对丢包率进行卡尔曼过滤分析预测网络负载状况,结合接收缓冲区大小设置来平滑发送速率,在分析TCP友好拥塞控制的基础上,提出了一种基于RTCP反馈的TCP友好拥塞控制机制。对该机制的TCP友好性、视频的回放质量进行了试验和结果分析。     

基于嵌入式CPU-GPU的高清鱼眼视频实时校正系统

在安防监控领域,需要鱼眼实时监控系统实现360°×180°大范围高质量无死角全景实时监控,现有的鱼眼校正系统存在成本较高,灵活性差,特别是清晰度不高和实时性差等方面的问题。针对如何提高全景高清鱼眼视频校正的实时性问题,提出了基于嵌入式平台STiH418的CPU-GPU高速通信协议和基于可编程着色器的嵌入式CPU-GPU内存共享方法,并利用GPU的纹理映射技术实现了全景高清鱼眼视频实时校正系统。实验结果表明,与相关校正系统相比,该系统很好地兼顾到算法效率、图像校正效果和完整性,可以完全满足360°×180°的全景高清（400万像素,2 048×2 048p30）鱼眼视频实时监控,而且与使用PC服务器相比嵌入式系统降低了系统整体成本,ARM CPU软件生成更新校正算法和可事时实时和事后的虚拟PTZ提高系统灵活性和稳定性,因此该系统具有很高的实用价值。     

大数据量实时视频流传输拥塞控制研究

大数据量实时视频流传输系统有广泛的应用前景。大数据量、高带宽要求的视频流传输中存在许多问题。网络的负载均衡、视频的回放质量都将严重影响整个网络的安全以及传输系统的应用。通过对丢包率进行卡尔曼过滤(Kalman Filter)分析预测网络负载状况、结合接收缓冲区大小设置来平滑发送速率,在分析TCP友好拥塞控制的基础上,提出了一种基于RTCP反馈的TCP友好(TCP-Friendly)对该机制的TCP友好性、视频的回放质量进行了实验和结果分析。     

机器人远程控制视频传输的实时性问题解决策略

分析了机器人远程控制中视频传输的延时问题。提出了一种混合的网络传输拓扑结构,并对传输速率的控制算法进行了移植和改进。利用NS模拟网络环境,在模拟网络中进行数据传输,验证了控制速率算法的可行性。试验结果表明在混合网络拓扑结构上使用控制速率的算法可以减少传输过程中的丢包率,提高视频传输的实时性。     

基于中国写意风格迁移的动漫视频生成模型

目前生成式对抗网络（GAN）已经被用于图像的动漫风格转换。然而,现有基于GAN的动漫生成模型主要以日本动漫和美国动漫为对象,集中在写实风格的提取与生成,很少关注到中国风动漫中写意风格的迁移,因此限制了GAN在国内广大动漫制作市场中的应用。针对这一问题,通过将中国写意风格融入到GAN模型,提出了一种新的中国风动漫生成式对抗网络模型CCGAN,用以自动生成具有中国写意风格的动漫视频。首先,通过在生成器中增加反向残差块,构造了一个轻量级的深度神经网络模型,以降低视频生成的计算代价。其次,为了提取并迁移中国写意风格中图像边缘锐利、内容构造抽象、描边线条具有水墨质感等性质,在生成器中构造了灰度样式损失和颜色重建损失,以约束真实图像和中国风样例图像在风格上的高层语义一致性,并且在判别器中构造了灰度对抗损失和边缘促进对抗损失,以约束重构图像与样例图像保持相同的边缘特性。最终,采用Adam算法最小化上述损失函数,从而实现风格迁移,并将重构图像组合为视频。实验结果表明,与目前最具代表性的风格迁移模型CycleGAN与CartoonGAN相比,所提CCGAN可从以《中国唱诗班》为例的中国风动漫中有效地学习到中国写意风格,同时显著降低了计算代价,适合于大批量动漫视频的快速生成。     

全互联视频会议实时数据传输路由算法设计

针对NAT环境下参与用户网络类型、网络状况往往不一致的情况,提出一种动态节点转发的实时多媒体数据传输技术.采用数据发送节点基于最优节点路由算法将数据发送给多媒体系统中的某些节点,然后由这些节点及下一批被选中接收的节点将数据经过多次转发,发送给系统中的其他节点,最终实现系统中各节点多媒体数据全互联.     

Optimal stream replication for video simulcasting

Video simulcasting enables a sender to generate replicated streams of different rates, serving receivers of diverse access bandwidths. As replication introduces noticeable redundancy, balancing bandwidth consumption with user satisfaction becomes a critical concern in simulcasting. This paper investigates the above issue; more explicitly, we seek answers to the following two questions: what is the number of streams that should be generated, and what is the bandwidth that should be allocated to each stream? We derive optimal and efficient solutions, and evaluate their performance under a variety of configurations. The results demonstrate that an optimal and adaptive bandwidth allocation significantly improves user satisfaction under stringent resource constraints, and an optimal choice of the stream number yields further improvements.     

基于内容的视频压缩改进算法

由于时间的连续性,视频前后帧图像有很大的相似性,视频图像前后帧存在着很大的时间冗余.针对视频图像的帧间冗余,在传统的视频编码算法基础上,提出一种基于匹配内容的视频压缩改进算法,根据实际视频图像的不同,选取不同的编码策略,并采用零树小波变换算法,使压缩比和实时性得到很大的提高.     

Biologically inspired fault-tolerant architectures for real-time control applications

Real-time control applications involve the interaction of multiple components. As systems become more complex their reliability tends to decrease, hence, fault tolerance must be incorporated to keep reliability within specified levels. A novel reconfiguration mechanism inspired by mechanisms that take place during the embryonic development of living beings is proposed in this paper. It is illustrated using an example that the rapid low-level fault-recovery characteristic of the embryonic system makes it a promising approach for real-time control applications.     

Feature-based real-time video stabilization for vehicle video recorder system

This paper presents a fast and effective method based on features to obtain real-time video stabilization for vehicle video recorder system. The corresponding feature points are first obtained from two consecutive frames and then optical flows are calculated based on these points. Next, the obtained optical flows are mapped to polar coordinates to obtain clusters and remove incorrect optical flows. These obtained clusters are used to evaluate the global motion and rotation angle. Finally, the obtained global motion and rotation angle are smoothed and then compensated to obtain the stabilized video. Experimental results show that the proposed method has good performance for video stabilization.     

Adaptive scene-detection algorithm for VBR video stream

Several scene-detection algorithms, which are only based on bit rate fluctuations, have been proposed. All of them are presented on the fixed thresholds, which are obtained by the empirical records of the video characteristics. Due to the sensitivity of these methods to the accuracy of the records, which are generally obtained by testing several values repeatedly, bad performance evaluation might be observed for the actual scene detection, especially for real-time video traffic. In this paper, we review the previous works in this area, and study the correlation between the scene duration and the scene change at the frame level, and simultaneously investigate the local statistical characteristics of scenes such as variance and peak bit rate etc. Based on this analysis, an effective decision function is first constructed for the scene segmentation. Then, we propose a scene-detection algorithm using the defined dynamic threshold model, which can capture the statistical properties of the scene changes. Experimental results using 15 variable bit rate MPEG video traces indicate good performances of the proposed algorithm with significantly improved scene-detection accuracy.     

Modeling video sources for real-time scheduling

What is the impact of the autocorrelation of variable-bit-rate (VBR) sources on real-time scheduling algorithms? Our results show that the impact of long term, or interframe, autocorrelation is negligible, while the impact of short term, or intraframe, autocorrelation can be significant. Such results are essentially independent of the video coding scheme employed. To derive these results, video sequences are modeled as a collection of stationary subsequences called scenes. Within a scene, a statistical model is derived for both the sequence of frames and of slices. The model captures the distribution and the autocorrelation function of real-time video data. In previous work, the pseudoperiodicity of the slice level auto-correlation function made it difficult to develop a simple yet accurate model. We present a generalization of previous methods that can easily capture this pseudoperiodicity and is suited for modeling a greater variety of autocorrelation functions. By simply tuning a few parameters, the model reproduces the statistic behavior of sources with different types and levels of correlation on both the frame and the slice level.     

Design of a high performance architecture for real-time enhancement of video stream captured in extremely low lighting environment

A high performance digital architecture for the implementation of a non-linear image enhancement technique is proposed in this paper. The image enhancement is based on a luminance dependent non-linear enhancement algorithm which achieves simultaneous dynamic range compression, colour consistency and lightness rendition. The algorithm provides better colour fidelity, enhances less noise, prevents the unwanted luminance drop at the uniform luminance areas, keeps the ‘bright’ background unaffected, and enhances the ‘dark’ objects in ‘bright’ background. The algorithm contains a large number of complex computations and thus it requires specialized hardware implementation for real-time applications. Systolic, pipelined and parallel design techniques are utilized effectively in the proposed FPGA-based architectural design to achieve real-time performance. Estimation techniques are also utilized in the hardware algorithmic design to achieve faster, simpler and more efficient architecture. The video enhancement system is implemented using Xilinx’s multimedia development board that contains a VirtexII-X2000 FPGA and it is capable of processing approximately 67 Mega-pixels (Mpixels) per second.     

A set packing inspired method for real-time junction train routing

Efficiently coordinating the often large number of interdependent, timetabled train movements on a railway junction, while satisfying a number of operational requirements, is one of the most important problems faced by a railway company. The most critical variant of the problem arises on a daily basis at major railway junctions where disruptions to rail traffic make the planned schedule/routing infeasible and rolling stock planners are forced to re-schedule/re-route trains in order to recover feasibility. The dynamic nature of the problem means that good solutions must be obtained quickly. In this paper we describe a set packing inspired formulation of this problem and develop a branch-and-price based solution approach. A real life test instance arising in Germany and supplied by the major German railway company, Deutsche Bahn, indicates the efficiency of the proposed approach by confirming that practical problems can be solved to within a few percent of optimality in reasonable time.     

Fuzzy-based rate control for real-time MPEG video

We propose a fuzzy logic-based control scheme for real-time motion picture expert group (MPEG) video to avoid long delay or excessive loss at the user-network interface (UNI) in an asynchronous transfer mode (ATM) network. The system consists of a shaper whose role is to smooth the MPEG output traffic to reduce the burstiness of the video stream. The input and output rates of the shaper buffer are controlled by two fuzzy logic-based controllers. To avoid a long delay at the shaper, the first controller aims to tune the output rate of the shaper in the video frame time scale based on the number of available transmission credits at the UNI and the occupancy of the shaper's buffer. Based on the average occupancy of the shaper's buffer and its variance, the second controller tunes the input rate to the shaper over a much larger time scale by applying a closed-loop MPEG encoding scheme. With this approach, the traffic enters the network at an almost constant bit rate (with a very small variation) allowing simple network management functions such as admission control and bandwidth allocation, while guaranteeing a relatively constant video quality since the encoding rate is changed only in critical periods when the shaper buffer “threatens” to overflow. The performance of the proposed scheme is evaluated through numerical tests on real video sequences     

A wavelet-VQ system for real-time video compression

This paper presents a scheme and its Field Programmable Gate Array (FPGA) implementation for a system based on combining the bi-dimensional discrete wavelet transformation (2D-DWT) and vector quantization (VQ) for image compression. The 2D-DWT works in a non-separable fashion using a parallel filter structure with distributed control to compute two resolution levels. The wavelet coefficients of the higher frequency sub-bands are vector quantized using multi-resolution codebook and those of the lower frequency sub-band at level two are scalar quantized and entropy encoded. VQ is carried out by self organizing feature map (SOFM) neural nets working at the recall phase. Codebooks are quickly generated off-line using the same nets functioning at the training phase. The complete system, including the 2D-DWT, the multi-resolution codebook VQ, and the statistical encoder, was implemented on a Xilinx Virtex 4 FPGA and is capable of performing real-time compression for digital video when dealing with grayscale 512 × 512 pixels images. It offers high compression quality (PSNR values around 35 dB) and acceptable compression rate values (0.62 bpp).

基于HBase的交通流数据实时存储系统

交通流数据具有多来源、高速率、体量大等特征,传统数据存储方法和系统暴露出扩展性弱和存储实时性低等问题.针对上述问题,设计并实现了一套基于HBase交通流数据实时存储系统.该系统采用分布式存储架构,通过前端的预处理操作对数据进行规范化整理,利用多源缓冲区结构对不同类型的流数据进行队列划分,并结合一致性哈希算法、多线程技术、行键优化设计等策略将数据并行存储到HBase集群服务器中.实验结果表明:该系统与基于Oracle的实时存储系统相比,其存储性能提升了3~5倍;与原生的HBase方法相比,其存储性能提升了2~3倍,并且具有良好的扩展性能.