一种支持H.264High Profile的高效可重构反变换VLSI结构

本文针对H.264 Fidelity Range Extensions(FRExt,High Profile)解码过程中扩展的所有变换,采用二维矩阵分解和基于矩阵运算提取公共因子的操作,利用通用运算单元来设计高效的可重构VLSI结构.该结构不但节省面积(可重构变换结构只消耗了4807门电路),并且具有高性能(采用TSM...     

Image and video processing using discrete fractional transforms

The mathematical transforms such as Fourier transform, wavelet transform and fractional Fourier transform have long been influential mathematical tools in information processing. These transforms process signal from time to frequency domain or in joint time–frequency domain. In this paper, with the aim to review a concise and self-reliant course, the discrete fractional transforms have been comprehensively and systematically treated from the signal processing point of view. Beginning from the definitions of fractional transforms, discrete fractional Fourier transforms, discrete fractional Cosine transforms and discrete fractional Hartley transforms, the paper discusses their applications in image and video compression and encryption. The significant features of discrete fractional transforms benefit from their extra degree of freedom that is provided by fractional orders. Comparison of performance states that discrete fractional Fourier transform is superior in compression, while discrete fractional cosine transform is better in encryption of image and video. Mean square error and peak signal-to-noise ratio with optimum fractional order are considered quality check parameters in image and video.     

Error concealment using affine transform for H.263 coded videotransmissions

A new error concealment method is proposed that uses motion estimation to consider actual motions, such as rotation, magnification, reduction, and parallel motion, in moving pictures. Since many videos include a variety of complex three-dimensional motions, the proposed method uses an affine transform to estimate the motion of lost data more accurately, thereby producing a higher peak signal-to-noise ratio value and better subjective video quality     

基于仿射模型的无人机视频实时压缩算法

在现有算法无法满足无人机视频实时压缩传输需求的应用背景下,该文提出一种新的低复杂度无人机(Unmanned Aerial Vehicle, UAV)视频实时压缩算法,利用无人机视频的背景平面性、运动一致性等特点,建立仿射模型描述视频的全局运动,并根据模型参数进行视频压缩。实验结果表明,与现有压缩算法H.264相比,该算法在保证压缩性能的前提下,大量减少了压缩时间,可满足大多数情况下无人机视频传输的质量要求和实时性要求。     

Temporal Anti‐aliasing of a Stereoscopic 3D Video

Frequency domain analysis is a fundamental procedure for understanding the characteristics of visual data. Several studies have been conducted with 2D videos, but analysis of stereoscopic 3D videos is rarely carried out. In this paper, we derive the Fourier transform of a simplified 3D video signal and analyze how a 3D video is influenced by disparity and motion in terms of temporal aliasing. It is already known that object motion affects temporal frequency characteristics of a time‐varying image sequence. In our analysis, we show that a 3D video is influenced not only by motion but also by disparity. Based on this conclusion, we present a temporal anti‐aliasing filter for a 3D video. Since the human process of depth perception mainly determines the quality of a reproduced 3D image, 2D image processing techniques are not directly applicable to 3D images. The analysis presented in this paper will be useful for reducing undesirable visual artifacts in 3D video as well as for assisting the development of relevant technologies.     

No-reference quality assessment of dynamic sports videos based on a spatiotemporal motion model

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.     

Real-time restoration of images degraded by uniform motion blur infoveal active vision systems

Foveated, log-polar, or space-variant image architectures provide a high resolution and wide field workspace, while providing a small pixel computation load. These characteristics are ideal for mobile robotic and active vision applications, but have been little used due to the general lack of image processing tools that are applicable to the log-polar coordinate system. Recently, we have described a generalization of the Fourier transform (the fast exponential chirp transform), which allows frame-rate computation of full-field two-dimensional (2-D) frequency transforms directly in log-polar coordinates. In the present work, we show that is possible to achieve full-frame image de-blur at frame rate on a standard "PC" platform, using these methods, we illustrate this idea with a Wiener filter based restoration technique. The main contribution of this note is the implementation of (space-variant) image de-blur directly in log-polar coordinates, using the exponential chirp transform. The results show reasonable quality of de-blur, and suggest that these methods are relevant to applications in mobile image processing platforms in which real-time motion deblur is important, and for which it is not desirable to use extensive or custom fabricated hardware.     

Object tracking in surveillance videos using compressed domain features from scalable bit-streams

Recent developments in the video coding technology brought new possibilities of utilising inherently embedded features of the encoded bit-stream in applications such as video adaptation and analysis. Due to the proliferation of surveillance videos there is a strong demand for highly efficient and reliable algorithms for object tracking. This paper presents a new approach for the fast compressed domain analysis utilising motion data from the encoded bit-streams in order to achieve low-processing complexity of object tracking in the surveillance videos. The algorithm estimates the trajectory of video objects by using compressed domain motion vectors extracted directly from standard H.264/MPEG-4 Advanced Video Coding (AVC) and Scalable Video Coding (SVC) bit-streams. The experimental results show comparable tracking precision when evaluated against the standard algorithms in uncompressed domain, while maintaining low computational complexity and fast processing time, thus making the algorithm suitable for real time and streaming applications where good estimates of object trajectories have to be computed fast.     

Early Coding Unit–Splitting Termination Algorithm for High Efficiency Video Coding (HEVC)

A new‐generation video coding standard, named High Efficiency Video Coding (HEVC), has recently been developed by JCT‐VC. This new standard provides a significant improvement in picture quality, especially for high‐resolution videos. However, one the most important challenges in HEVC is time complexity. A quadtree‐based structure is created for the encoding and decoding processes and the rate‐distortion (RD) cost is calculated for all possible dimensions of coding units in the quadtree. This provides a high encoding quality, but also causes computational complexity. We focus on a reduction scheme of the computational complexity and propose a new approach that can terminate the quadtree‐based structure early, based on the RD costs of the parent and current levels. Our proposed algorithm is compared with HEVC Test Model version 10.0 software and a previously proposed algorithm. Experimental results show that our algorithm provides a significant time reduction for encoding, with only a small loss in video quality.     

Blind video quality assessment based on multilevel video perception

Blind video quality assessment (VQA) metrics predict the quality of videos without the presence of reference videos. This paper proposes a new blind VQA model based on multilevel video perception, abbreviated as MVP. The model fuses three levels of video features occurring in natural video scenes to predict video quality: natural video statistics (NVS) features, global motion features and motion temporal correlation features. They represent video scene characteristics, video motion types, and video temporal correlation variations. In the process of motion feature extraction, motion compensation filtering video enhancement is adopted to highlight the motion characteristics of videos so as to improve the perceptual correlations of the video features. The experimental results on the LIVE and CSIQ video databases show that the predicted video scores of the new model are highly correlated with human perception and have low root mean square errors. MVP obviously outperforms state-of-art blind VQA metrics, and particularly demonstrates competitive performance even compared against top-performing full reference VQA metrics.     

LMM-based frame-level rate control for H.264/AVC high-definition video coding

Accurate distribution modeling for the DCT coefficients is greatly important for us to analyze the rate–distortion (R–D) behavior of video encoders. From the experiment, we observed that most of the existing models, paying more attention to the standard-definition (SD) videos, tend not to work well for high-definition (HD) videos. Motivated by this, in this paper, we address the statistical characteristics of DCT coefficients of HD videos coded by H.264/AVC. The contributions of this paper are threefold: first, Laplacian Mixture Model (LMM) is proposed to model the residues instead of using Laplacian or Cauchy distribution; second, the LMM-based analytic rate and distortion models are derived; third, building on the proposed rate and distortion models, a frame-level rate control algorithm is developed. Experimental results show that the proposed rate control method achieves a PSNR improvement of up to 0.85 dB compared with the rate control scheme adopted in the H.264 reference software [1]. Apart from the average visual quality improvement, the temporal visual quality fluctuation is reduced by 17%.     

Enhanced zonal search algorithm for motion estimation in depth-map coding

This paper presents a novel motion estimation scheme for depth-map coding. Depth map has a couple of different characteristics compared with the common color scene, and therefore, careful investigation of the conventional fast motion search schemes is required. In this paper, I first provide the necessity of the depth-oriented motion search scheme based on experiments and then analyze the problems of the conventional methods. On the basis of the analysis, I propose to use the initial position refinement step during a motion search. In detail, the modification of the one-at-a-time search scheme is proposed for improvement of the prediction accuracy, and an initial direction selection and a 1-bit transform scheme follow for complexity reduction. The experimental results show that the proposed scheme can achieve coding gains of more than 3% on average over the state-of-the-art zonal search methods while keeping the overall complexity increase low.     

Parallel pipeline implementation of wavelet transforms

Wavelet transforms have been one of the important signal processing developments in the last decade, especially for applications such as time-frequency analysis, data compression, segmentation and vision. Although several efficient implementations of wavelet transforms have been derived, their computational burden is still considerable. The paper describes two generic parallel implementations of wavelet transforms, based on the pipeline processor farming methodology, which have the potential to achieve real-time performance. Results show that the parallel implementation of the oversampled wavelet transform achieves virtually linear speedup, while the parallel implementation of the discrete wavelet transform (DWT) also outperforms the sequential version, provided that the filter order is large. The DWT parallelisation performance improves with increasing data length and filter order, while the frequency-domain implementation performance is independent of wavelet filter order. Parallel pipeline implementations are currently suitable for processing multidimensional images with data length at least 512 pixels     

Automating the modeling and optimization of the performance ofsignal transforms

Fast implementations of discrete signal transforms, such as the discrete Fourier transform (DFT), the Walsh-Hadamard transform (WHT), and the discrete trigonometric transforms (DTTs), can be viewed as factorizations of their corresponding transformation matrices. A given signal transform can have many different factorizations, with each factorization represented by a unique but mathematically equivalent formula. When implemented in code, these formulas can have significantly different running times on the same processor, sometimes differing by an order of magnitude. Further, the optimal implementations on various processors are often different. Given this complexity, a crucial problem is automating the modeling and optimization of the performance of signal transform implementations. To enable computer modeling of signal processing performance, we have developed and analyzed more than 15 feature sets to describe formulas representing specific transforms. Using some of these features and a limited set of training data, we have successfully trained neural networks to learn to accurately predict performance of formulas with error rates less than 5%. In the direction of optimization, we have developed a new stochastic evolutionary algorithm known as STEER that finds fast implementations of a variety of signal transforms. STEER is able to optimize completely new transforms specified by a user. We present results that show that STEER can find discrete cosine transform formulas that are 10-20% faster than what a dynamic programming search finds     

A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand

This paper proposes a novel real-time electromyogram (EMG) pattern recognition for the control of a multifunction myoelectric hand from four channel EMG signals. To extract a feature vector from the EMG signal, we use a wavelet packet transform that is a generalized version of wavelet transform. For dimensionality reduction and nonlinear mapping of the features, we also propose a linear-nonlinear feature projection composed of principal components analysis (PCA) and a self-organizing feature map (SOFM). The dimensionality reduction by PCA simplifies the structure of the classifier and reduces processing time for the pattern recognition. The nonlinear mapping by SOFM transforms the PCA-reduced features into a new feature space with high class separability. Finally, a multilayer perceptron (MLP) is used as the classifier. Using an analysis of class separability by feature projections, we show that the recognition accuracy depends more on the class separability of the projected features than on the MLP's class separation ability. Consequently, the proposed linear-nonlinear projection method improves class separability and recognition accuracy. We implement a real-time control system for a multifunction virtual hand. Our experimental results show that all processes, including virtual hand control, are completed within 125 ms, and the proposed method is applicable to real-time myoelectric hand control without an operational time delay.     

Removing interference components in time–frequency representations using morphological operators

Time–frequency representations have been of great interest in the analysis and classification of non-stationary signals. The use of highly selective transformation techniques is a valuable tool for obtaining accurate information for studies of this type. The Wigner-Ville distribution has high time and frequency selectivity in addition to meeting some interesting mathematical properties. However, due to the bi-linearity of the transform, interference terms emerge when the transform is applied over multi-component signals. In this paper, we propose a technique to remove cross-components from the Wigner-Ville transform using image processing algorithms. The proposed method exploits the advantages of non-linear morphological filters, using a spectrogram to obtain an adequate marker for the morphological processing of the Wigner-Ville transform. Unlike traditional smoothing techniques, this algorithm provides cross-term attenuations while preserving time–frequency resolutions. Moreover, it could also be applied to distributions with different interference geometries. The method has been applied to a set of different time–frequency transforms, with promising results.     

An HMM based analysis framework for semantic video events

Semantic video analysis plays an important role in the field of machine intelligence and pattern recognition. In this paper, based on the Hidden Markov Model （HMM）, a semantic recognition framework on compressed videos is proposed to analyze the video events according to six low-level features. After the detailed analysis of video events, the pattern of global motion and five features in foreground-the principal parts of videos, are employed as the observations of the Hidden Markov Model to classify events in videos. The applications of the proposed framework in some video event detections demonstrate the promising success of the proposed framework on semantic video analysis.     

适于微小型飞行器平台的稳像算法

为解决微小型飞行器由于机械振动、气流扰动等原因引起的图像高频抖动,设计了一种适用于该平台的稳像算法。采用基于生物视觉的匹配方法估计帧间运动矢量,建立了图像参数传递的数学模型;结合微小型飞行器的运动特点,提出了带约束的交互式多模型卡尔曼滤波方法(CIMMKF),针对绝对帧位移曲线和旋转角度滤波,引入硬约束条件减小模型不准确性产生的误差,再通过软约束平滑硬约束带来的局部跳变求得图像合适的校正量。最后,给出了一种新颖的微小型飞行器平台稳像算法性能的评估方法。实验结果表明,该稳像算法能够适应飞行器多种状态的交替改变,有效减小滤波延迟,去除高频抖动,保留主动运动,使稳定后图像质量满足观察要求,具有图像信息保留程度高、速度快的特点,尤其适用于微小型飞行器实时视频稳定。     

Fast computation of the discrete Walsh and Hadamard transforms

The discrete Walsh and Hadamard transforms are often used in image processing tasks such as image coding, pattern recognition, and sequency filtering. A new discrete Walsh transform (DWT) algorithm is derived in which a modified form of the DWT relation is decomposed into smaller-sized transforms using vectorized quantities. A new sequency-ordered discrete Hadamard transform (DHAT) algorithm is also presented. The proposed approach results in more regular algorithms requiring no independent data swapping and fewer array-index updating and bit-reversal operations. An analysis of the computational complexity and the execution time performance are provided. The results are compared with those of the existing algorithms