Fast prediction unit selection method for HEVC intra prediction based on salient regions

In order to reduce the computational complexity of the high efficiency video coding(HEVC) standard, a new algorithm for HEVC intra prediction, namely, fast prediction unit(PU) size selection method for HEVC based on salient regions is proposed in this paper. We first build a saliency map for each largest coding unit(LCU) to reduce its texture complexity. Secondly, the optimal PU size is determined via a scheme that implements an information entropy comparison among sub-blocks of saliency maps. Finally, we apply the partitioning result of saliency map on the original LCUs, obtaining the optimal partitioning result. Our algorithm can determine the PU size in advance to the angular prediction in intra coding, reducing computational complexity of HEVC. The experimental results show that our algorithm achieves a 37.9% reduction in encoding time, while producing a negligible loss in Bjontegaard delta bit rate(BDBR) of 0.62%.     

An algorithm for coding video signal based on 3-D wavelet transformation

This paper presents an algorithm for coding video signal based on 3-D wavelet transformation. When the frame order t of a video signal is replaced by order 2, the video signal can be looked as a block in 3-D space. After splitting the block into smaller sub-blocks, imitate the method of 2-D wavelet transformation for images, we can transform the sub-blocks with 3-D wavelet. Most of video signal energy is in the decomposed low-frequency sub-bands. These sub-bands affect the visual quality of the video signal most. Quantizing different sub-bands with different precision and then entropy encoding each sub-band, we can eliminate inter- and intra-frame redundancy of the video signal and compress data. Our simulation experiments show that this algorithm can achieve very good result.     

Coding-oriented multi-view video color correction

Color inconsistency between views is an important problem to be solved in multi-view video applications, such as free viewpoint television and other three-dimensional video systems. In this paper, by combining with multi-view video coding, a coding-oriented multi-view video color correction method is proposed. We first separate foreground and background in first Group Of Pictures （GOP） by using SKIP coding mode. Then by transferring means and standard deviations in backgrounds, color correction is performed for each frame in GOP, and multi-view video coding is performed and used to renew the backgrounds. Experimental results ances in color correction and multi-view video show the proposed method can obtain better performcoding.     

Region-of-interest based rate control for UAV video coding

To meet the requirement of high-quality transmission of videos captured by unmanned aerial vehicles (UAV) with low bandwidth, a novel rate control (RC) scheme based on region-of-interest (ROI) is proposed. First, the ROI information is sent to the encoder with the latest high efficient video coding (HEVC) standard to generate an ROI map. Then, by using the ROI map, bit allocation methods are developed at frame level and large coding unit (LCU) level, to avoid inaccurate bit allocation produced by camera movement. At last, by using a better robustness R-λ model, the quantization parameter (QP) for each LCU is calculated. The experimental results show that the proposed RC method can get a lower bitrate error and a higher quality for reconstructed video by choosing appropriate pixel weight on the HEVC platform.     

A ROBUST ADAPTIVE VIDEO ENCODER BASED ON HUMAN VISUAL MODEL

A Robust Adaptive Video Encoder (RAVE) based on human visual model is proposed.The encoder combines the best features of Fine Granularity Scalabla (FGS) coding,frame-dropping coding,video redundancy coding,and human visual model.According to packet loss and available bandwidth of the network,the encoder adjust the output bit rate by jointly adapting quantization step-size instructed by human visual model,rate shaping,and periodically inserting key frame.The proposed encoder is implemented based on MPEG-4 encoder and is compared with the case of a conventional FGS algorithm.It is shown that RAVE is a very efficient robust videl encoder that provides improved visual quality for the receiver and consumes equal or less network resource.Results are confirmed by subjective tests and simulation tests.     

A FINE GRANULAR JOINT SOURCE CHANNEL CODING METHOD

An improved FGS (Fine Granular Scalability) coding method is proposed in this letter,which is based on human visual characteristics.This method adjusts FGS coding frame rate according to the evaluation of video sequences so as to improve the coding efficiency and subject perceived quality of reconstructed images.Finally,a fine granular joint source channel coding is proposed based on the source coding method,which not only utilizes the network resources efficiently,but guarantees the reliable transmission of video information.     

Novel two-layer motion estimation for video coding

In this paper,we proposed a novel Two-layer Motion Estimation （TME） which searches motion vectors on two layers with partial distortion measures in order to reduce the overwhelming computational complexity of Motion Estimation （ME） in video coding.A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer.It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame.The proposed TME performs a coarse search on the first layer to identify the small region in which the best candidate block is likely to be positioned and then perform local refined search on the next layer to pick the best candidate block in the located small area.The key feature of TME is its flexibility of mixing with any fast search algorithm.Experimental results on a wide variety of video sequences show that the proposed algorithm has achieved both fast speed and good motion prediction quality when compared to well known as well as the state-of-the-art fast block matching algorithms.     

Evolution Handoff Strategy for Real-Time Video Transmission over Practical Cognitive Radio Networks

The transmission delay of realtime video packet mainly depends on the sensing time delay(short-term factor) and the entire frame transmission delay(long-term factor).Therefore,the optimization problem in the spectrum handoff process should be formulated as the combination of microscopic optimization and macroscopic optimization.In this paper,we focus on the issue of combining these two optimization models,and propose a novel Evolution Spectrum Handoff(ESH)strategy to minimize the expected transmission delay of real-time video packet.In the microoptimized model,considering the tradeoff between Primary User’s(PU’s) allowable collision percentage of each channel and transmission delay of video packet,we propose a mixed integer non-linear programming scheme.The scheme is able to achieve the minimum sensing time which is termed as an optimal stopping time.In the macro-optimized model,using the optimal stopping time as reward function within the partially observable Markov decision process framework,the EHS strategy is designed to search an optimal target channel set and minimize the expected delay of packet in the long-term real-time video transmission.Meanwhile,the minimum expected transmission delay is obtained under practical cognitive radio networks’ conditions,i.e.,secondary user’s mobility,PU’s random access,imperfect sensing information,etc..Theoretical analysis and simulation results show that the ESH strategy can effectively reduce the transmission delay of video packet in spectrum handoff process.     

An Algorithm of Wavelet Image Fusion Based on Visual Saliency

Considering the characteristics of wavelet transform and human visual system saliency, a novel image fusion algorithm was proposed. Based on multi-scale wavelet decomposition, visual saliency model had been introduced into calculating the intensity, direction and texture global integrated saliency of each pixel at each level frequency component. And it adaptively acquired the weighted fusing coefficients. Experiments show that this algorithm can achieve more clear minutia. Comparing with regional feature-based wavelet fusion algorithm, it improves fusion performance of human vision and objective evaluation. It is also benefit to detection and recognition.     

AVS 3D Video Coding Technology and System

Following the success of the audio video standard(AVS) for 2D video coding,in 2008,the China AVS workgroup started developing 3D video(3DV) coding techniques.In this paper,we discuss the background,technical features,and applications of AVS 3DV coding technology.We introduce two core techniques used in AVS 3DV coding:inter-view prediction and enhanced stereo packing coding.We elaborate on these techniques,which are used in the AVS real-time 3DV encoder.An application of the AVS 3DV coding system is presented to show the great practical value of this system.Simulation results show that the advanced techniques used in AVS 3DV coding provide remarkable coding gain compared with techniques used in a simulcast scheme.     

Human centered perceptual adaptation for video coding

Traditional visual saliency based video compression methods try to encode the image with higher quality in the region of saliency. However, the saliency feature changes according to persons, viewpoints, and distances. In this paper, we propose to apply a technique of human centered perceptual computation to improve video coding in the region of human centered perception. To detect the region of interest (ROI) of human body, upper body, frontal face, and profile face, we construct Harr and histogram of oriented gradients features based combo of detectors to analyze a video in the first frame (intra-frame). From the second frame (inter-frame) onward, the optical flow image is computed in the ROI area of the first frame. The optical flow in human centered ROI is then used for macroblock (MB) quantization adjustment in H.264/AVC. For each MB, the quantization parameter (QP) is optimized with density value of optical flow image. The QP optimization process is based on a MB mapping model, which can be calculated by an inverse of the inverse tangent function. The Lagrange multiplier in the rate distortion optimization is also adapted so that the MB distortion at human centered region is minimized. We apply our technique to the H.264 video encoder to improve coding visual quality. By evaluating our scheme with the H.264 reference software, our results show that the proposed algorithm can improve the visual quality of ROI by about 1.01 dB while preserving coding efficiency.     

Video abstraction based on the visual attention model and online clustering

With the fast evolution of digital video, research and development of new technologies are greatly needed to lower the cost of video archiving, cataloging and indexing, as well as improve the efficiency and accessibility of stored video sequences. A number of methods to respectively meet these requirements have been researched and proposed. As one of the most important research topics, video abstraction helps to enable us to quickly browse a large video database and to achieve efficient content access and representation. In this paper, a video abstraction algorithm based on the visual attention model and online clustering is proposed. First, shot boundaries are detected and key frames in each shot are extracted so that consecutive key frames in a shot have the same distance. Second, the spatial saliency map indicating the saliency value of each region of the image is generated from each key frame and regions of interest (ROI) is extracted according to the saliency map. Third, key frames, as well as their corresponding saliency map, are passed to a specific filter, and several thresholds are used so that the key frames containing less information are discarded. Finally, key frames are clustered using an online clustering method based on the features in ROIs. Experimental results demonstrate the performance and effectiveness of the proposed video abstraction algorithm.     

Extraction technique of region of interest from stereoscopic video

A feature fusion approach is presented to extract the region of interest (ROI) from the stereoscopic video. [0]Based on human vision system (HVS), the depth feature, the color feature and the motion feature are chosen as vision features. [0]The algorithm is shown as follows. Firstly, color saliency is calculated on superpixel scale. Color space distribution of the superpixel and the color difference between the superpixel and background pixel are used to describe color saliency and color salient region is detected. Then, the classic visual background extractor (Vibe) algorithm is improved from the update interval and update region of background model. The update interval is adjusted according to the image content. The update region is determined through non-obvious movement region and background point detection. So the motion region of stereoscopic video is extracted using improved Vibe algorithm. The depth salient region is detected by selecting the region with the highest gray value. Finally, three regions are fused into final ROI. Experiment results show that the proposed method can extract ROI from stereoscopic video effectively. In order to further verify the proposed method, stereoscopic video coding application is also carried out on the joint model (JM) encoder with different bit allocation in ROI and the background region.     

No-reference quality assessment of HEVC video streams based on visual memory modelling

Providing adequate Quality of Experience (QoE) to end-users is crucial for streaming service providers. In this paper, in order to realize automatic quality assessment, a No-Reference (NR) bitstream Human-Vision-System-(HVS)-based video quality assessment (VQA) model is proposed. Inspired by discoveries from the neuroscience community, which suggest there is a considerable overlap between active areas of the brain when engaging in video quality assessment and saliency detection tasks, saliency maps are used in the proposed method to improve the quality assessment accuracy. To this end, saliency maps are first generated from features extracted from the HEVC bitstream. Then, saliency map statistics are employed to create a model of visual memory. Finally, a support vector regression pipeline learns an estimate of the video quality from the visual memory, saliency, and frame features. Evaluations on SJTU dataset indicate that the proposed bitstream based no-reference video quality assessment algorithm achieves a competitive performance.     

基于内容复杂度的视频编码码率控制

该文根据线性模型提出信源内容复杂度的一种度量方式,并以此进行帧级比特分配和宏块级的视频码率控制。仿真表明该方法与MPEG2中的TM5相比,能比较显著地提高信噪比,实现更加准确的码率控制,同时可以在帧间和帧内获得更一致的视觉效果。该算法在MPEG4及低码率视频编码中也是适用的。     

基于视觉显著计算的视频流媒体渐进式表达方法

为了有效解决视频流媒体传输网络带宽、播出视频质量和用户实时性访问之间的矛盾,本文提出了一种基于视觉显著计算的视频流媒体渐进式表达方法.在视频内容分析和理解的基础上,首先进行场景分类和视觉敏感区域提取;然后根据编码信息确定视频序列中各帧的重要性,估计帧内片层数据重要性;最后基于视觉显著计算的结果提出一种适应网络带宽和质量可伸缩的视频流媒体渐进式表达方法.采用中粒度质量可伸缩（MGS）编码,在模拟网络测试平台上分别针对集中式和分散式视觉敏感区域视频序列进行实验研究,实验结果验证了本文提出的基于视觉显著计算的视频流媒体渐进式表达方法的正确性和有效性.     

基于三维变换域频谱差的视频显著性检测算法

显著性区域检测是计算机视觉的重要课题,对视频质量评价和感知视频编码算法优化也至关重要.大多显著性检测算法不能权衡准确率和复杂度,限制了它们在视频预处理和实时处理中的应用.提出了一种基于三维变换域频谱差(3DTDSD)的快速视频显著性检测算法,分别以关键帧及其前一帧为中心建立一定数量图像帧的滑动窗,得到两组3D视频体,用傅里叶变换将两组视频变换到三维频域,两组三维数据之间的差值经过反变换得到显著性图,最后通过连通分析、阈值判断等得到显著区域.频域算法具有运算速度快的特点,实验对比和算法复杂度分析证明了该算法的有效性和快速性.     

Visual saliency guided video compression algorithm

Recently Saliency maps from input images are used to detect interesting regions in images/videos and focus on processing these salient regions. This paper introduces a novel, macroblock level visual saliency guided video compression algorithm. This is modelled as a 2 step process viz. salient region detection and frame foveation. Visual saliency is modelled as a combination of low level, as well as high level features which become important at the higher-level visual cortex. A relevance vector machine is trained over 3 dimensional feature vectors pertaining to global, local and rarity measures of conspicuity, to yield probabilistic values which form the saliency map. These saliency values are used for non-uniform bit-allocation over video frames. To achieve these goals, we also propose a novel video compression architecture, incorporating saliency, to save tremendous amount of computation. This architecture is based on thresholding of mutual information between successive frames for flagging frames requiring re-computation of saliency, and use of motion vectors for propagation of saliency values.     

Scale-aware saliency for application to frame rate upconversion

Our understanding of human visual perception has been paramount in the development of tools for digital video processing. For this reason, saliency detection, i.e., the determination of visual importance in a scene, has come to the forefront in recent literature. In the proposed work, a new method for scale-aware saliency detection is introduced. Scale determination is afforded through a scale-space model utilizing color and texture cues. Scale information is fed back to a discriminant saliency engine by automatically tuning center-surround parameters through a soft weighting. Excellent results are demonstrated for the proposed method through its performance against a database of measured human fixations. Further evidence of the proposed algorithm's performance is demonstrated through an application to frame rate upconversion. The ability of the algorithm to detect salient objects at multiple scales allows for class-leading performance both objectively, in terms of peak signal-to-noise ratio/structural similarity index, and subjectively. Finally, the need for operator tuning of saliency parameters is dramatically reduced by the inclusion of scale information. The proposed method is well suited for any application requiring automatic saliency determination for images or video.     

Foveation scalable video coding with automatic fixation selection

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.