基于Wiener滤波的快速视频去雾算法

雾霾环境会极大的降低视频中事物的能见度。针对于目前去雾算法存在运算复杂度大、处理时间长的缺点,提出一种基于Wiener滤波的快速去雾算法,首先获取暗原色模型,通过Wiener滤波自适应获取透射率分布图并进行初级及深度去雾,最后通过自适应对数变换进行亮度调整,得到去雾后图像。实验证明,相比现有的先进方法,本文的算法具有处理速度快和去雾效果好的优点。     

基于雾线先验的时空关联约束视频去雾算法

现有视频去雾算法由于缺少对视频结构关联约束和帧间一致性分析,容易导致连续帧去雾结果在颜色和亮度上存在突变,同时去雾后的前景目标边缘区域也容易出现退化现象。针对上述问题,该文提出一种基于雾线先验的时空关联约束视频去雾算法,通过引入每帧图像在空间邻域中具有的结构关联性和时间邻域中具有的连续一致性,提高视频去雾算法的求解准确性和鲁棒性。算法首先使用暗通道先验估计每帧图像的大气光向量,并结合雾线先验求取初始透射率图。然后引入加权最小二乘边缘保持平滑滤波器对初始透射率图进行空间平滑,消除奇异点和噪声对估计结果的影响。进一步利用相机参数刻画连续帧间透射率图的时序变化规律,对独立求取的每帧透射率图进行时序关联修正。最后根据雾图模型获得最终的视频去雾结果。定性和定量的对比实验结果表明,该算法下视频去雾结果的帧间过渡更加自然,同时对每一帧图像的色彩还原更加准确,图像边缘的细节信息显示也更加丰富。     

Modeling and Evaluating Non-shared Memory CELL/BE Type Multi-core Architectures for Local Image and Video Processing

Local processing, which is a dominant type of processing in image and video applications, requires a huge computational power to be performed in real-time. However, processing locality, in space and/or in time, allows to exploit data parallelism and data reusing. Although it is possible to exploit these properties to achieve high performance image and video processing in multi-core processors, it is necessary to develop suitable models and parallel algorithms, in particular for non-shared memory architectures. This paper proposes an efficient and simple model for local image and video processing on non-shared memory multi-core architectures. This model adopts a single program multiple data approach, where data is distributed, processed and reused in an optimal way, regarding the data size, the number of cores and the local memory capacity. The model was experimentally evaluated by developing video local processing algorithms and programming the Cell Broadband Engine multi-core processor, namely for advanced video motion estimation and in-loop deblocking filtering. Furthermore, based on these experiences it is also addressed the main challenges of vectorization, and the reduction of branch mispredictions and computational load imbalances. The limits and advantages of the regular and adaptive algorithms are also discussed. Experimental results show the adequacy of the proposed model to perform local video processing, and that real-time is achieved even to process the most demanding parts of advanced video coding. Full-pixel motion estimation is performed over high resolution video (720×576 pixels) at a rate of 30 frames per second, by considering large search areas and five reference frames.     

视频图像去雾算法的自适应机制设计及FPGA加速实现

该文提出了一种自适应图像去雾算法,充分考虑不同复杂场景下的图像特征,建立了算法的自适应机制。该机制包含对图像是否有雾、是否为天空区域、滤波器尺寸等的自适应调整,解决了传统图像去雾算法在深度断层处可能产生的光晕效应等问题。该文同时对上述自适应图像去雾算法进行FPGA加速实现,实验结果表明,该文算法在XC7K325T型号FPGA视频处理平台上可以满足对1080P@60Hz视频去雾的实时性要求。对于大多数轻雾或浓雾场景,该文算法去雾后图像色彩自然无过饱和,全局对比度和饱和度提升比率均值为0.309和0.994,相比于本领域其他去雾算法优势明显。     

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Compression of captured video frames is crucial for saving the power in wireless capsule endoscopy (WCE). A low complexity encoder is desired to limit the power consumption required for compressing the WCE video. Distributed video coding (DVC) technique is best suitable for designing a low complexity encoder. In this technique, frames captured in RGB colour space are converted into YCbCr colour space. Both Y and CbCr representing luma and chroma components of the Wyner–Ziv (WZ) frames are processed and encoded in existing DVC techniques proposed for WCE video compression. In the WCE video, consecutive frames exhibit more similarity in texture and colour properties. The proposed work uses these properties to present a method for processing and encoding only the luma component of a WZ frame. The chroma components of the WZ frame are predicted by an encoder–decoder based deep chroma prediction model at the decoder by matching luma and texture information of the keyframe and WZ frame. The proposed method reduces the computations required for encoding and transmitting of WZ chroma component. The results show that the proposed DVC with a deep chroma prediction model performs better when compared to motion JPEG and existing DVC systems for WCE at the reduced encoder complexity.     

Motion Estimation Based on Spline Interpolation in H.264/AVC Video Coder for Videoconferencing Application: Performance Versus Computation Load

The powerful H.264/AVC video coder involves a large encoding computational cost than the existing video standards due mainly to the motion-compensated estimation scheme based on a full search of multiple reference frames in the sequence. This strategy decreases the residual errors of the predicted frames and may improve the performance of the video coder. However a great number of computations are usually wasted without improving significantly the quality of the decoded video mostly in videoconferencing applications. To reduce the encoding computational load and preserve the performance of the video coder, this paper proposes to substitute the motion-compensated estimation method implemented in H.264/AVC by a temporal spline interpolation. Simulations on several test sequences show that important encoding saving times are achieved with a competitive quality of the decoded video compared to the exhaustive search of multiple reference frames in the H.264/AVC video coder.     

基于残差网络的无监督角膜视频分割算法

基于角膜形变计算出一系列生物力学特性参数是训练早期圆锥角膜分类模型的数据基础,因此圆锥角膜轮廓分割的精确性直接影响着早期圆锥角膜分类模型的准确性。本文提出了一种基于残差网络的无监督角膜视频分割方法。通过统一的网格化采样提取一组锚点被同序列视频帧所共用,从而减小网络模型学习特征表示的计算量并且提高了计算效率。同时设计了一个正则化分支对原有的视频集进行相似性转换来解决可能存在的退化解问题。与已有的无监督视频分割任务相比,本实验模型使用了少量的训练数据,但却取得了更高的分割精度和计算效率。     

Zero-quantised discrete cosine transform prediction technique for video encoding

A new analytical model to eliminate redundant discrete cosine transform (DCT) and quantisation (Q) computations in block-based video encoders is proposed. The dynamic ranges of the quantised DCT coefficients are analysed, then a threshold scheme is derived to determine whether the DCT and Q computations can be skipped without video quality degradation. In addition, fast DCT/inverse DCT (IDCT) algorithms are presented to implement the proposed analytical model. The proposed analytical model is compared with other comparable analytical models reported in the literature. Both the theoretical analysis and experimental results demonstrate that the proposed analytical model can greatly reduce the computational complexity of video encoding without any performance degradation and outperforms other analytical models     

A progressive CNN in-loop filtering approach for inter frame coding

Convolutional Neural Network (CNN) structures have been designed for in-loop filtering to improve video coding performance. These CNN models are usually trained through learning the correlations between the reconstructed and the original frames, which are then applied to every single reconstructed frame to improve the overall video quality. This direct model training and deployment strategy is effective for intra coding since a locally optimal model is sufficient. However, when applied to inter coding, it causes over-filtering because the intertwined reference dependencies across inter frames are not taken into consideration. To address this issue, existing methods usually resort to the Rate–Distortion Optimization (RDO) to selectively apply the CNN model, but fail to address the limitation of using a local CNN model. In this paper, we propose a progressive approach to train and incorporate the CNN-based in-loop filters to work seamlessly with video encoders. First, we develop a progressive training method to obtain the inter model. Using transfer learning, reconstructed frames using the CNN model are progressively involved back into the training of the CNN model itself, to simulate the reference dependencies in inter coding. Next, we design a frame-level model selection strategy for the high-bitrate coding where the over-filtering effect is diluted. Experimental results show that the proposed method outperforms the RDO method that utilizes only local model. Proposed approach also achieves comparable coding performance but with less computational complexity when integrating our progressive model into the RDO scheme.     

A fast image super-resolution algorithm using an adaptive Wiener filter.

A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.     

基于FPGA的视频实时去雾算法及其硬件实现

单幅图像去雾技术虽然已经取得较大的进展,但是算法较为复杂,运行时间较长.为了实现视频实时去雾,以硬件实现为目的,对暗通道先验算法进行改进,降低其时间复杂度.提出了一种暗通道图优化方法,保留了图像的边缘信息,消除了光晕效应,省去了透射率细化的复杂操作;提出了适应于硬件实现的大气光值估计和调节及透射率补偿方法,解决了视频帧间闪烁及天空等明亮区域的色彩失真问题.基于现场可编程门阵列(FPGA)对所提出算法进行了硬件实现.结果表明,该算法可以实时处理帧速为60 f/s、分辨率为1 920×1 080的视频图像,相比传统去雾算法速度更快,去雾质量更高.     

Level-set-based motion estimation algorithm for multiple reference frame motion estimation

Motion estimation (ME) has a variety of applications in image processing, pattern recognition, target tracking, and video compression. In modern video compression standards such as H.264/AVC and HEVC, multiple reference frame ME (MRFME) is adopted to reduce the temporal redundancy between successive frames in a video sequence. In MRFME, the motion search process is conducted using additional reference frames, thereby obtaining better prediction signal as compared to single reference frame ME (SRFME). However, its high computational complexity makes it difficult to be utilized in real-world applications. In order to reduce the computational complexity of MRFME, this paper proposes a level-set-based ME algorithm (LSME) without any penalty in the rate-distortion (RD) performance. First, the proposed algorithm partitions the motion search space into multiple level sets based on a rate constraint. The proposed algorithm then controls the ME process on the basis of the predetermined level sets. Experimental results show that the proposed algorithm reduces the ME time by up to 83.46% as compared to the conventional full search (FS) algorithm.     

Video compressive sensing reconstruction via long- short-term double-pattern prediction

The compressive sensing technology has a great potential in high-dimensional vision processing. The existing video reconstruction methods utilize the multihypothesis prediction to derive the residual sparse model from key frames. However, these methods cannot fully utilize the temporal correlation among multiple frames. Therefore, this paper proposes the video compressive sensing reconstruction via long-short-term double-pattern prediction, which consists of four main phases:the first phase reconstructs each frame independently; the second phase adaptively updates multiple reference frames; the third phase selects the hypothesis matching patches from current reference frames; the fourth phase obtains the reconstruction results by using the patches to build the residual sparse model. The experimental results demonstrate that as compared with the state-of-the-art methods, the proposed methods can obtain better prediction accuracy and reconstruction quality for video compressive sensing.     

Gradient Fusion Method for Night Video Enhancement

To resolve video enhancement problems, a novel method of gradient domain fusion wherein gradient domain frames of the background in daytime video are fused with nighttime video frames is proposed. To verify the superiority of the proposed method, it is compared to conventional techniques. The implemented output of our method is shown to offer enhanced visual quality.     

Online Dictionary Learning Based Intra-frame Video Coding

In this paper, we propose an online learning based intra-frame video coding approach, exploiting the texture sparsity of natural images. The proposed method is capable of learning the basic texture elements from previous frames with convergence guaranteed, leading to effective dictionaries for sparser representation of incoming frames. Benefiting from online learning, the proposed online dictionary learning based codec (ODL codec) is able to achieve a goal that the more video frames are being coded, the less non-zero coefficients are required to be transmitted. Then, these non-zero coefficients for image patches are further quantized and coded combined with dictionary synchronization. The experimental results demonstrate that the number of non-zero coefficients of each frame decreases rapidly while more frames are encoded. Compared to the off-line mode training, the proposed ODL codec, learning from video on the fly, is able to reduce the computational complexity with fast convergence. Finally, the rate distortion performance shows improvement in terms of PSNR compared with the K-SVD dictionary based compression and H.264/AVC for intra-frame video at low bit rates.     

利用视频非局部相似性的分布式压缩感知重构

为了提高分布式视频压缩感知（Distributed Video Compressive Sensing,DVCS）的率失真性能,仅利用稀疏先验知识不能很好地保护视频帧的边缘与纹理细节,本文提出利用视频非局部相似性形成正则化项融入联合重构模型以有效去除边缘与纹理区域的模糊和块效应现象。仿真实验表明,本文所提出的联合重构算法可有效地改善主客观视频重构质量,能以一定计算复杂度为代价提高分布式视频压缩感知系统的率失真性能。      

Key-frame selection for video summarization: an approach of multidimensional time series analysis

This paper presents a novel method of key-frame selection for video summarization based on multidimensional time series analysis. In the proposed scheme, the given video is first segmented into a set of sequential clips containing a number of similar frames. Then the key frames are selected by a clustering procedure as the frames closest to the cluster centres in each resulting video clip. The proposed algorithm is implemented experimentally on a wide range of testing data, and compared with state-of-the-art approaches in the literature, which demonstrates excellent performance and outperforms existing methods on frame selection in terms of fidelity-based metric and subjective perception.     

Performance modeling of MPEG‐4 video streaming over IEEE 802.11 using distribution coordination function

Delivering video streaming over wireless Internet is becoming increasingly popular. However, most of the research studies focused on the modeling analysis of system performance such as saturation throughput and channel utilization. Perceived quality of video streaming cannot be assessed solely based on the results of analytical models. In this paper, we propose a model to assess the perceived quality of MPEG‐4 video streaming over IEEE 802.11 distribution coordination function (DCF)‐based wireless local area networks. The analysis of our proposed model considers not only effects of losses such as collision loss from channel access competition but also wireless loss caused by wireless interferences. Moreover, the impact of the loss of specific MPEG‐4 video frames is also taken into account in the performance analysis. The model was validated by comparing our performance results with results obtained from simulation and analytical models. The results show that our proposed model is able to predict the perceived quality of MPEG‐4 video streaming over DCF‐based WLAN more accurately than other models. Copyright © 2010 John Wiley & Sons, Ltd.     

基于FPGA的图像高速去雾实时系统设计实现

雾霾使光照条件恶劣,导致采集的视频图像失真.为了解决这个问题,本文采用Altera公司的Cyclone IV系列现场可编程门阵列(Field-Programmable Gate Array,FPGA)芯片作为核心,设计了支持多种分辨率的图像高速去雾实时系统.通过RAM的乒乓操作缓存高速数据流,并利用流水线处理的优势实现了限制对比度自适应直方图均衡化(Contrast Limit-ed Adaptive His-togram Equalization,CLAHE)算法的流程.实验结果表明,该系统能处理高达75 帧/秒的视频图像,具有良好的实时去雾功能.     

Video compressed sensing reconstruction based on structural group sparsity and successive approximation estimation model

The existing video compressed sensing (CS) algorithms for inconsistent sampling ignore the joint correlations of video signals in space and time, and their reconstruction quality and speed need further improvement. To balance reconstruction quality with computational complexity, we introduce a structural group sparsity model for use in the initial reconstruction phase and propose a weight-based group sparse optimization algorithm acting in joint domains. Then, a coarse-to-fine optical flow estimation model with successive approximation is introduced for use in the interframe prediction stage to recover non-key frames through alternating optical flow estimation and residual sparse reconstruction. Experimental results show that, compared with the existing algorithms, the proposed algorithm achieves a peak signal-to-noise ratio gain of 1–3 dB and a multi-scale structural similarity gain of 0.01–0.03 at a low time complexity, and the reconstructed frames not only have good edge contours but also retain textural details.