Content-based hybrid error concealment approach for packet video communication over the noisy channels

Video compression makes the encoded video stream more vulnerable to the channel errors so that, the quality of the received video is exposed to severe degradation when the compressed video is transmitted over the error-prone environments. Therefore, it is necessary to apply error concealment (EC) techniques in the decoder to improve the quality of the received video. In this regard, an Adaptive Content-based EC Approach (ACBECA) is proposed in this paper, which exploits both the spatial and temporal correlations within the video sequences for the EC purpose. The proposed approach adaptively utilizes two EC techniques, including new spatial-temporal error concealment (STEC) technique, and a temporal error concealment (TEC) technique, to recover the lost regions of the frame. The STEC technique proposed in this paper is established on the basis of non-Local Means concept and tries to recover each lost macroblock (MB) as the weighted average of the similar MBs in the reference frame, whereas the TEC technique recovers the motion vector of the lost MB adaptively by analyzing the behavior of the MB in the frame. The decision on temporally or spatially reconstructing the degraded frames is made dynamically according to the content of the degraded frame (i.e., structure or texture), type of the error and also block loss rate (BLR). Compared with the state-of-the-art EC techniques, the simulation results indicate the superiority of the ACBECA in terms of both the objective and subjective quality assessments.

     

A Novel Approach to Improve the Speech Intelligibility Using Fractional Delta-amplitude Modulation Spectrogram

Abstract

Speech enhancement is an interesting research area that aims at improving the quality and intelligibility of the speech that is affected by the additive noises, such as airport noise, train noise, restaurant noise, and so on. The presence of these background noises degrades the comfort of listening of the end user. This article proposes a speech enhancement method that uses a novel feature extraction which removes the noise spectrum from the noisy speech signal using a novel fractional delta-AMS (amplitude modulation spectrogram) feature extraction and the D-matrix feature extraction method. The fractional delta-AMS feature extraction strategy is the modification of the delta-AMS with the fractional calculus that increases the sharpness of the feature extraction. The extracted features from the frames are used to determine the optimal mask of all the frames of the noisy speech signal and the mask is employed for training the deep belief neural networks (DBN). The two metrics root mean square error (RMSE) and perceptual evaluation of speech quality (PESQ) are used to evaluate the method. The proposed method yields a better value of PESQ at all level of noise and RMSE decreases with increased noise level.     

基于像素同龄组和相邻组的绝缘子去噪方法

针对红外图像带有脉冲噪声和高斯噪声的特点,提出了一种新的去噪方法。首先根据像素同龄组所含的个数来确定脉冲噪声和信号区域,然后利用像素相邻组判断目标边缘,在保持边缘和有用信号区域不变的同时,利用同龄组内的像素平均值对其它像素进行去噪处理。实验表明,该方法在信噪比上比其它方法要优越,并且能较好地保留边缘等细节信息。     

Video sequence feature extraction and segmentation using likelihood regression model

The development of digital technology is utilized by people to capture and share video frames. At present, rather than capturing images, people are interested in recording video footage for exploring information. Here, retrieval of video from large databases is challenging due to the continuous frame count. To overcome these challenges associated with the retrieval of video from available databases, this research proposed a likelihood-based regression approach for video processing. To improve the retrieval accuracy of video sequences, the proposed method utilizes a likelihood estimation technique integrated with a regression model. The likelihood estimate measures the pixel level roughly for estimating the pixel range, after which the regression approach measures the pixel level for transforming certainly blurred and unwanted pixels. In the proposed likelihood regression approach, the video is converted into a video frame and stored in a database. Query frames are taken into account by the generated database depending on the features which are used for a given video to be retrieved. The significant video retrieval performance obtained from the simulation results for the proposed likelihood-based regression model shows that the proposed model performs well over the other state-of-the-art techniques.

     

高效的I帧分区错误隐藏方法

针对现有I帧错误隐藏方法不能平衡恢复图像质量与算法复杂度的问题,提出了一种高效的I帧分区错误隐藏方法。首先,利用视频帧之间的运动相关性将丢失宏块分为运动宏块和静止宏块。对于静止宏块,采用帧拷贝法进行掩盖;对于运动宏块,再根据其周围正确解码宏块的纹理信息将其分为平滑块和纹理块。对平滑块采用双线性插值法进行恢复;对纹理块利用比较精细的指数分布权重的加权模板匹配(WTE)法进行掩盖。实验结果表明,与WTE算法相比,所提方法的峰值信噪比(PSNR)平均提高了2.6 dB,计算复杂度平均降低了90%。对于场景连续的具有不同特征和分辨率的视频序列,所提方法都具有一定的适用性。     

High capacity information hiding for privacy protection in digital video files

In this paper, a framework to hide privacy in video is proposed based on data hiding principals. A novel data hiding technique is proposed and implemented to hide the original frame into the in-painted one. The proposed hiding technique is carried out in the discrete wavelet transform domain of the cover video. The proposed technique is embedding video into video. Furthermore, the proposed data hiding method can blindly reconstruct the original frame from the fake one. Experimental results showed that the proposed method can successfully hide the complete frames of the original video into their corresponding in-painted ones that are as large as themselves. Simple visual inspection of the results showed that the quality of the stego-frames maintain very high (above 45 dB) while providing an acceptable visual quality for the retrieved original frames.

     

A robust multimedia surveillance system for people counting

Closed circuit television cameras (CCTV) are widely used in monitoring. This paper presents an intelligent CCTV crowd counting system based on two algorithms that estimate the density of each pixel in each frame and use it as a basis for counting people. One algorithm uses scale-invariant feature transform (SIFT) features and clustering to represent pixels of frames (SIFT algorithm) and the other uses features from accelerated segment test (FAST) corner points with SIFT features (SIFT-FAST algorithm). Each algorithm is designed using a novel combination of pixel-wise, motion-region, grid map, background segmentation using Gaussian mixture model (GMM) and edge detection. A fusion technique is proposed and used to validate the accuracy by combining the result of the algorithms at frame level. The proposed system is more practical than the state of the art regression methods because it is trained with a small number of frames so it is relatively easy to deploy. In addition, it reduces the training error, set-up time, cost and open the door to develop more accurate people detection methods. The University of California (UCSD) and Mall datasets have been used to test the proposed algorithms. The mean deviation error, mean squared error and the mean absolute error of the proposed system are less than 0.1, 16.5 and 3.1, respectively, for the Mall dataset and less than 0.07, 5.5 and 1.9, respectively, for UCSD dataset.

     

Multiclass SVM based adaptive filter for removal of high density impulse noise from color images

This paper proposes a multiclass support vector machine (SVM) based adaptive filter for removal of impulse noise from color images. The quality of the image gets degraded due to the presence of impulse noise. As a result, the homogeneity amongst the pixels gets distorted that needs to be restored. The feature set comprising of prediction error, difference between the median value and the center pixel; the median value in the kernel under operation has been used during this study. The pixel of test image is processed using adaptive window based filter that depends on the associated class assigned at the testing phase. The baseline system has been designed using modified histogram based fuzzy color filter (MHFC) technique. Four set of experiments have been carried out on a large database to validate the proposed method. The performance of the technique have been evaluated using peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM). The results suggest that for fixed valued impulse noise, the proposed filter performs better than the MHFC in case of high density impulse noise (>45%). However, for random valued impulse noise the proposed filter outperforms the MHFC based method for both low and high density of noise. The objective analysis suggests that there is ∼3 dB improvement in PSNR as compared to the MHFC based method for high density of impulse noise. The results of SSIM along with visual observations indicate that the image details are maintained significantly in the proposed technique as compared to existing methods.     

低复杂度空域错误隐藏算法

在无线网等不可靠信道传输视频中,针对包含边缘信息的宏块丢失带来的错误,提出了一种低复杂度的空间域错误隐藏算法,该算法根据周围已经接收到的宏块特征,估算丢失宏块中边缘方向,并按照边缘方向使用周围临近像素值进行插值,恢复出包含边缘的宏块;在插值过程中提出并采用了一种快捷的方向插值法。实验结果表明,该算法能有效地掩盖丢失宏块,比常规方法具有更好的掩盖效果与实时性。     

面向视频感知的静电力触觉渲染方法

针对视觉障碍的人获取视频等数字媒体信息受限的问题, 为扩展视频等数字媒体信息的触觉感知通道,提出一种面向视频感知的静电力触觉渲染方法。首先,采用基于像素点的视频帧处理算法,根据手指触摸位置获取当前视频帧的目标像素点,然后将目标像素点彩色信息从RGB模型转换为HSI模型,利用像素点色调分量来映射静电力激励信号频率参量,结合像素点亮度和饱和度分量来映射静电力激励信号幅度参量,合成静电力触觉激励信号,实现对实时视频的触觉渲染和感知。最后,设计动态色彩感知实验和亮度辨识感知实验,结果表明,该方法可实现对视频中物体信息的触觉感知,动态识别平均正确率达90.6%,色彩辨识平均正确率达69.4%,亮度辨识平均正确率达80.0%,所提方法能有效提取视频中的动态特征信息,增强视频触觉渲染的实时性。     

图像脉冲噪声的自适应长距离相关迭代滤波法

提出一种基于局部极值噪声检测的自适应长距离相关迭代滤波算法.该算法首先采用局部极值法进行噪声检测,然后在一定的搜寻范围内计算信号点与噪声点的背景均方误差值,并以该背景均方误差值为基础采用自适应加权法进行滤波,最后将这一滤波过程进行迭代计算.实验结果表明,该算法滤波效果优于传统的滤波算法,它可以有效地去除图像中的脉冲噪声,并较好地保持图像细节信息,在噪声密度很大的情况下也表现出很好的滤波性能.     

自适应权重更新的轻量级视频目标分割算法

目的 针对现有视频目标分割（video object segmentation,VOS）算法不能自适应进行样本权重更新,以及使用过多的冗余特征信息导致不必要的空间与时间消耗等问题,提出一种自适应权重更新的轻量级视频目标分割算法。方法 首先,为建立一个具有较强目标判别性的算法模型,所提算法根据提取特征的表征质量,自适应地赋予特征相应的权重;其次,为了去除冗余信息,提高算法的运行速度,通过优化信息存储策略,构建了一个轻量级的记忆模块。结果 实验结果表明,在公开数据集DAVIS2016 （densely annotated video segmentation）和DAVIS2017上,本文算法的区域相似度与轮廓准确度的均值J&F分别达到了85.8%和78.3%,与对比的视频目标分割算法相比具有明显的优势。结论 通过合理且无冗余的历史帧信息利用方式,提升了算法对于目标建模的泛化能力,使目标掩码质量更高。     

图像去噪混合滤波方法

传统均值滤波和中值滤波对高斯型噪声和椒盐型噪声有着不同的滤波特性。实际滤波时，由于图像往往会受到两种不同性质噪声的同时干扰，因此，单独采用中值滤波或均值滤波都不会达到最好的去噪效果，为了能同时对两种不同性质的噪声进行滤除，现提出了一种新的混合滤波算法，该算法首先利用局部阈值把受高斯型噪声污染的像素和受脉冲型噪声污染的像素区别开来，然后对受高斯噪声污染的像素采用均值滤波算法，而对受椒盐噪声污染的像素则采用中值滤波算法进行去噪。仿真结果证明，该方法更具有实用性和有效性。     

Ultra-HEVC using frame frequency error optimization technique for IPTV realization

Internet Protocol Television (IPTV) is an emerging network application in the internet world. One of the most reliable networks is IPTV which gives high speed for internet services. As IPTV offers many live services on user demand and it has many advantages. But still, some problem exists in the existing implementation such as degradation of quality and delay while maintaining limited frames and efficient bandwidth consumption over the network channel. The efficient bandwidth utilization is a major issue in IPTV platforms. Integrating the video processing on network platform is the challenging task in video on demand (VoD) application. This paper overcomes the drawbacks of existing IPTV by using Frame Frequency Error Optimization (FFEO) based HEVC approach which is called as U-HEVC. The FFEO method upgrades the video quality by interpolation of frames. U-HEVC delivers 50% better compression similar to the existing HEVC standard and it also provides better visual quality at half the bit rates. The Analysis of proposed U-HEVC attain better results compared to existing HEVC compression algorithms that higher number of packets get affected at different bit rate levels. In HEVC the Frame loss of 1 Mbps is 0.38%, 2 Mbps is 0.46%, 4 Mbps is 0.63% and 8 Mbps is 0.94%. When compared to the U-HEVC the Frame loss is somewhat high in HEVC. This paper presents the studies on IPTV environment based on U-HEVC using frame frequency error optimization technique.

     

Iterative deviation filter for fixed-valued impulse noise removal

An iterative deviation filter for fixed-valued impulse noise removal is proposed, with the aim to overcome the defects of existing filters, and further improve the denoising performance. In the proposed filter, a noise detection method based on the extreme intensity values and the deviation of neighbor pixels is proposed, i.e., the pixels with the extreme intensity and differ greatly from the mean of neighbor pixels, are identified as noises. A noise removal method based on the minimum deviation of neighbor pixels is proposed, i.e., the intensity of one neighbor noise free pixel, which is closest to the mean of neighbor noise free pixels, is used as estimated intensity of noisy pixel under consideration. Furthermore, the noise removal strategy performs iteratively and takes full advantage of the previous denoising results. Simulation results show that the proposed method has better denoising performance than the existing distinguished filters in terms of visual representation, peak signal to noise ratio and structural similarity index.

     

实时视觉目标跟踪与视频对象分割多任务框架

目的 针对视觉目标跟踪（video object tracking,VOT）和视频对象分割（video object segmentation,VOS）问题,研究人员提出了多个多任务处理框架,但是该类框架的精确度和鲁棒性较差。针对此问题,本文提出一个融合多尺度上下文信息和视频帧间信息的实时视觉目标跟踪与视频对象分割多任务的端到端框架。方法 文中提出的架构使用了由空洞深度可分离卷积组成的更加多尺度的空洞空间金字塔池化模块,以及具备帧间信息的帧间掩模传播模块,使得网络对多尺度目标对象分割能力更强,同时具备更好的鲁棒性。结果 本文方法在视觉目标跟踪VOT-2016和VOT-2018数据集上的期望平均重叠率（expected average overlap,EAO）分别达到了0.462和0.408,分别比SiamMask高了0.029和0.028,达到了最先进的结果,并且表现出更好的鲁棒性。在视频对象分割DAVIS（densely annotated video segmentation）-2016和DAVIS-2017数据集上也取得了有竞争力的结果。其中,在多目标对象分割DAVIS-2017数据集上,本文方法比SiamMask有更好的性能表现,区域相似度的杰卡德系数的平均值J_M和轮廓精确度的F度量的平均值F_M分别达到了56.0和59.0,并且区域和轮廓的衰变值J_D和F_D都比SiamMask中的低,分别为17.9和19.8。同时运行速度为45帧/s,达到了实时的运行速度。结论 文中提出的融合多尺度上下文信息和视频帧间信息的实时视觉目标跟踪与视频对象分割多任务的端到端框架,充分捕捉了多尺度上下文信息并且利用了视频帧间的信息,使得网络对多尺度目标对象分割能力更强的同时具备更好的鲁棒性。     

Dissimilar pixel counting based impulse detector for two-phase mixed noise removal

Mixed noise is a challenging noise model due to its statistical complexity. A new two-phase denoising method based on an impulse detector using dissimilar pixel counting is proposed in this paper. This method consists of two stages: detection and filtering. For the detection phase, average difference scheme is proposed to distinguish whether two neighboring pixels are similar or not, and then the number of dissimilar pixels is compared with a threshold to locate the outlier point in noisy image. An iterative framework is used for detection accuracy with the least numbers of iteration. For the filtering phase, an extended trilateral filter is used to remove the mixture of Gaussian and impulse noise, which are treated differently depending on the guidance matrix from the detection phase. Extensive experimental results demonstrate that the proposed method exhibits better noise detection capability and outperforms many existing two-phase mixed noise removal methods in both quantitative evaluation and visual quality.

     

基于深度体素流的模糊视频插帧方法

针对视频运动模糊严重影响插帧效果的情况,提出了一种新型的模糊视频插帧方法。首先,提出一种多任务融合卷积神经网络,该网络结构由两个模块组成：去模糊模块和插帧模块。其中,去模糊模块采用残差块堆叠的深度卷积神经网络（CNN）,提取并学习深度模糊特征以实现两帧输入图像的运动模糊去除;插帧模块用于估计帧间的体素流,所得体素流将用于指导像素进行三线性插值以合成中间帧。其次,制作了大型模糊视频仿真数据集,并提出一种先分后合、由粗略至细致的训练策略,实验结果表明该策略促进了多任务网络有效收敛。最后,对比前沿的去模糊和插帧算法组合,实验指标显示所提方法合成中间帧时峰值信噪比最少提高1.41 dB,结构相似性提升0.020,插值误差降低1.99。视觉对比及重制序列展示表明,所提模型对于模糊视频有着显著的帧率上转换效果,即能够将两帧模糊视频帧端对端重制为清晰且视觉连贯的三帧视频帧。     

Key Frame Extraction of Surveillance Video based on Moving Object Detection and Image Similarity

For the traditional method to extract the surveillance video key frame, there are problems of redundant information, substandard representative content and other issues. A key frame extraction method based on motion target detection and image similarity is proposed in this paper. This method first uses the ViBe algorithm fusing the inter-frame difference method to divide the original video into several segments containing the moving object. Then, the global similarity of the video frame is obtained by using the peak signal to noise ratio, the local similarity is obtained through the SURF feature point, and the comprehensive similarity of the video image is obtained by weighted fusion of them. Finally, the key frames are extracted from the critical video sequence by adaptive selection threshold. The experimental results show that the method can effectively extract the video key frame, reduce the redundant information of the video data, and express the main content of the video concisely. Moreover, the complexity of the algorithm is not high, so it is suitable for the key frame extraction of the surveillance video.