Foveation-Based Error Resilience and Unequal Error Protection over Mobile Networks

By exploiting new human-machine interface techniques, such as visual eyetrackers, it should be possible to develop more efficient visual multimedia services associated with low bandwidth, dynamic channel adaptation and robust visual data transmission. In this paper, we introduce foveation-based error resilience and unequal error protection techniques over highly error-prone mobile networks. Each frame is spatially divided into foveated and background layers according to perceptual importance. Perceptual importance is determined either through an eye tracker or by manually selecting a region of interest. We attempt to improve reconstructed visual quality by maintaining the high visual source throughput of the foveated layer using foveation-based error resilience and error correction using a combination of turbo codes and ARQ (automatic reQuest). In order to alleviate the degradation of visual quality, a foveation based bitstream partitioning is developed. In an effort to increase the source throughput of the foveated layer, we develop unequal delay-constrained ARQ (automatic reQuest) and rate compatible punctured turbo codes where the punctual pattern of RCPC (rate compatible punctured convolutional) codes in H.223 Annex C is used. In the simulation, the visual quality is significantly increased in the area of interest using foveation-based error resilience and unequal error protection; (as much as 3 dB FPSNR (foveal peak signal to noise ratio) improvement) at 40% packet error rate. Over real-fading statistics measured in the downtown area of Austin, Texas, the visual quality is increased up to 1.5 dB in PSNR and 1.8 dB in FPSNR at a channel SNR of 5 dB.     

Design of linear equalizers optimized for the structural similarity index.

We propose an algorithm for designing linear equalizers that maximize the structural similarity (SSIM) index between the reference and restored signals. The SSIM index has enjoyed considerable application in the evaluation of image processing algorithms. Algorithms, however, have not been designed yet to explicitly optimize for this measure. The design of such an algorithm is nontrivial due to the nonconvex nature of the distortion measure. In this paper, we reformulate the nonconvex problem as a quasi-convex optimization problem, which admits a tractable solution. We compute the optimal solution in near closed form, with complexity of the resulting algorithm comparable to complexity of the linear minimum mean squared error (MMSE) solution, independent of the number of filter taps. To demonstrate the usefulness of the proposed algorithm, it is applied to restore images that have been blurred and corrupted with additive white gaussian noise. As a special case, we consider blur-free image denoising. In each case, its performance is compared to a locally adaptive linear MSE-optimal filter. We show that the images denoised and restored using the SSIM-optimal filter have higher SSIM index, and superior perceptual quality than those restored using the MSE-optimal adaptive linear filter. Through these results, we demonstrate that a) designing image processing algorithms, and, in particular, denoising and restoration-type algorithms, can yield significant gains over existing (in particular, linear MMSE-based) algorithms by optimizing them for perceptual distortion measures, and b) these gains may be obtained without significant increase in the computational complexity of the algorithm.     

Complex Wavelet Structural Similarity: A New Image Similarity Index

We introduce a new measure of image similarity called the complex wavelet structural similarity (CW-SSIM) index and show its applicability as a general purpose image similarity index. The key idea behind CW-SSIM is that certain image distortions lead to consistent phase changes in the local wavelet coefficients, and that a consistent phase shift of the coefficients does not change the structural content of the image. By conducting four case studies, we have demonstrated the superiority of the CW-SSIM index against other indices (e.g., Dice, Hausdorff distance) commonly used for assessing the similarity of a given pair of images. In addition, we show that the CW-SSIM index has a number of advantages. It is robust to small rotations and translations. It provides useful comparisons even without a preprocessing image registration step, which is essential for other indices. Moreover, it is computationally less expensive.     

Unequal Power Allocation for JPEG Transmission Over MIMO Systems

With the introduction of multiple transmit and receive antennas in next generation wireless systems, real-time image and video communication are expected to become quite common, since very high data rates will become available along with improved data reliability. New joint transmission and coding schemes that explore advantages of multiple antenna systems matched with source statistics are expected to be developed. Based on this idea, we present an unequal power allocation scheme for transmission of JPEG compressed images over multiple-input multiple-output systems employing spatial multiplexing. The JPEG-compressed image is divided into different quality layers, and different layers are transmitted simultaneously from different transmit antennas using unequal transmit power, with a constraint on the total transmit power during any symbol period. Results show that our unequal power allocation scheme provides significant image quality improvement as compared to different equal power allocations schemes, with the peak-signal-to-noise-ratio gain as high as 14 dB at low signal-to-noise-ratios.      

Analysis of multichannel narrow-band filters for image texturesegmentation

A model for texture analysis and segmentation using multiple oriented channel filters is analyzed in the general framework. Several different arguments are applied leading to the conclusion that the two-dimensional Gabor filters possess strong optimality properties for this task. Properties of the multiple-channel segmentation approach are analyzed. In particular, perturbations of textures from an ideal model are found to have important effects on the segmentation that can usually be ameliorated by simply preceding the segmentation process by a logarithmic operation and using a low-pass postfilter prior to making region assignments. The difficult problems of space-variant textures and multiple component textures are also considered. Local spatial frequency estimation approaches are suggested that use the responses as constraints in estimating the locally emergent texture frequencies. Complex texture aggregates containing multiple shared frequency components can be analyzed if the textures are distinct and few in number     

Nonlinear image estimation using piecewise and local image models

We introduce a new approach to image estimation based on a flexible constraint framework that encapsulates meaningful structural image assumptions. Piecewise image models (PIMs) and local image models (LIMs) are defined and utilized to estimate noise-corrupted images, PIMs and LIMs are defined by image sets obeying certain piecewise or local image properties, such as piecewise linearity, or local monotonicity. By optimizing local image characteristics imposed by the models, image estimates are produced with respect to the characteristic sets defined by the models. Thus, we propose a new general formulation for nonlinear set-theoretic image estimation. Detailed image estimation algorithms and examples are given using two PIMs: piecewise constant (PICO) and piecewise linear (PILI) models, and two LIMs: locally monotonic (LOMO) and locally convex/concave (LOCO) models. These models define properties that hold over local image neighborhoods, and the corresponding image estimates may be inexpensively computed by iterative optimization algorithms. Forcing the model constraints to hold at every image coordinate of the solution defines a nonlinear regression problem that is generally nonconvex and combinatorial. However, approximate solutions may be computed in reasonable time using the novel generalized deterministic annealing (GDA) optimization technique, which is particularly well suited for locally constrained problems of this type. Results are given for corrupted imagery with signal-to-noise ratio (SNR) as low as 2 dB, demonstrating high quality image estimation as measured by local feature integrity, and improvement in SNR.     

Embedded foveation image coding

The human visual system (HVS) is highly space-variant in sampling, coding, processing, and understanding. The spatial resolution of the HVS is highest around the point of fixation (foveation point) and decreases rapidly with increasing eccentricity. By taking advantage of this fact, it is possible to remove considerable high-frequency information redundancy from the peripheral regions and still reconstruct a perceptually good quality image. Great success has been obtained previously by a class of embedded wavelet image coding algorithms, such as the embedded zerotree wavelet (EZW) and the set partitioning in hierarchical trees (SPIHT) algorithms. Embedded wavelet coding not only provides very good compression performance, but also has the property that the bitstream can be truncated at any point and still be decoded to recreate a reasonably good quality image. In this paper, we propose an embedded foveation image coding (EFIC) algorithm, which orders the encoded bitstream to optimize foveated visual quality at arbitrary bit-rates. A foveation-based image quality metric, namely, foveated wavelet image quality index (FWQI), plays an important role in the EFIC system. We also developed a modified SPIHT algorithm to improve the coding efficiency. Experiments show that EFIC integrates foveation filtering with foveated image coding and demonstrates very good coding performance and scalability in terms of foveated image quality measurement.     

An information fidelity criterion for image quality assessment using natural scene statistics.

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Traditionally, image QA algorithms interpret image quality as fidelity or similarity with a "reference" or "perfecft" image in some perceptual space. Such "full-referenc" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by arbitrary signal fidelity criteria. In this paper, we approach the problem of image QA by proposing a novel information fidelity criterion that is based on natural scene statistics. QA systems are invariably involved with judging the visual quality of images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of natural signals, that is, pictures and videos of the visual environment. Using these statistical models in an information-theoretic setting, we derive a novel QA algorithm that provides clear advantages over the traditional approaches. In particular, it is parameterless and outperforms current methods in our testing. We validate the performance of our algorithm with an extensive subjective study involving 779 images. We also show that, although our approach distinctly departs from traditional HVS-based methods, it is functionally similar to them under certain conditions, yet it outperforms them due to improved modeling. The code and the data from the subjective study are available at.     

Visual pattern image coding

A novel framework for digital image compression called visual pattern image coding, or VPIC, is presented. In VPIC, set of visual-patterns is defined independent of the images to be coded. Each visual pattern is a subimage of limited spatial support that is visually meaningful to a normal human observer. The patterns are used as a basis for efficient image representation; since it is assumed that the images to be coded are natural optical images to be viewed by human observers, visual pattern design is developed using relevant psychophysical and physiological data. Although VPIC bears certain resemblances to block truncation (BTC) and vector quantification (VQ) image coding, there are important differences. First, there is no training phase required: the visual patterns derive from models of perceptual mechanisms; second, the assignment of patterns to image regions is not based on a standard (norm) error criterion; expensive search operations are eliminated