A compression method for a massive image data set in image-based rendering

In image-based rendering with adjustable illumination, the data set contains a large number of pre-captured images under different sampling lighting directions. Instead of individually compressing each pre-captured image, we propose a two-level compression method. Firstly, we use a few spherical harmonic (SH) coefficients to represent the plenoptic property of each pixel. The classical discrete summation method for extracting SH coefficient requires that the sampling lighting directions should be uniformly distributed on the whole spherical surface. It cannot handle the case that the sampling lighting directions are irregularly distributed. A constrained least-squares algorithm is proposed to handle this case. Afterwards, embedded zero-tree wavelet coding is used for removing the spatial redundancy in SH coefficients. Simulation results show our approach is much superior to the JPEG, JPEG2000, MPEG2, and 4D wavelet compression method. The way to allow users to interactively control the lighting condition of a scene is also discussed.     

JPEG Pleno: Providing representation interoperability for holographic applications and devices

Guaranteeing interoperability between devices and applications is the core role of standards organizations. Since its first JPEG standard in 1992, the Joint Photographic Experts Group (JPEG) has published several image coding standards that have been successful in a plethora of imaging markets. Recently, these markets have become subject to potentially disruptive innovations owing to the rise of new imaging modalities such as light fields, point clouds, and holography. These so‐called plenoptic modalities hold the promise of facilitating a more efficient and complete representation of 3D scenes when compared to classic 2D modalities. However, due to the heterogeneity of plenoptic products that will hit the market, serious interoperability concerns have arisen. In this paper, we particularly focus on the holographic modality and outline how the JPEG committee has addressed these tremendous challenges. We discuss the main use cases and provide a preliminary list of requirements. In addition, based on the discussion of real‐valued and complex data representations, we elaborate on potential coding technologies that range from approaches utilizing classical 2D coding technologies to holographic content‐aware coding solutions. Finally, we address the problem of visual quality assessment of holographic data covering both visual quality metrics and subjective assessment methodologies.     

基于结构信息和像素失真的客观图像质量评价

为了度量多种失真类型的图像质量,根据人类视觉系统(HVS)对图像空域结构信息高度敏感和任一类型的失真都会产生像素失真理论,提出一种基于结构信息和像素失真的无参考的质量评价方法.该方法利用色彩信息提取能够表征图像结构信息的视觉内容结构图,并加权像素失真来度量图像质量,同时对部分失真类型进行修正.该方法不涉及任何参数设置也无需训练过程.实验结果表明,该方法能够较好地评价白噪声、JPEG压缩、高斯模糊、JPEG2000压缩和FastFading等失真图像的质量,并与主观评价方法有较好的一致性.     

Image representation by compressive sensing for visual sensor networks

This paper addresses the image representation problem in visual sensor networks. We propose a new image representation method for visual sensor networks based on compressive sensing (CS). CS is a new sampling method for sparse signals, which is able to compress the input data in the sampling process. Combining both signal sampling and data compression, CS is more capable of image representation for reducing the computation complexity in image/video encoder in visual sensor networks where computation resource is extremely limited. Since CS is more efficient for sparse signals, in our scheme, the input image is firstly decomposed into two components, i.e., dense and sparse components; then the dense component is encoded by the traditional approach (JPEG or JPEG 2000) while the sparse component is encoded by a CS technique. In order to improve the rate distortion performance, we leverage the strong correlation between dense and sparse components by using a piecewise autoregressive model to construct a prediction of the sparse component from the corresponding dense component. Given the measurements and the prediction of the sparse component as initial guess, we use projection onto convex set (POCS) to reconstruct the sparse component. Our method considerably reduces the number of random measurements needed for CS reconstruction and the decoding computational complexity, compared to the existing CS methods. In addition, our experimental results show that our method may achieves up to 2 dB gain in PSNR over the existing CS based schemes, for the same number of measurements.     

Image content analysis for sector-wise JPEG fragment classification

In this paper, we propose a sector-wise JPEG fragment classification approach to classify normal and erroneous JPEG data fragments with the minimum size of 512 bytes per fragment. Our method is based on processing each read-in sector of 512 bytes with using the DCT coefficient analysis methods for extracting the features of visual inconsistencies. The classification is conducted before the inverse DCT and can be performed simultaneously with JPEG decoding. The contributions of this work are two-folds: (1) a sector-wise JPEG erroneous fragment classification approach is proposed (2) new DCT coefficient analysis methods are introduced for image content analysis. Testing results on a variety of erroneous fragmented and normal JPEG files prove the strength of this operator for the purpose of forensics analysis, data recovery and abnormal fragment inconsistencies classification and detection. Furthermore, the results also show that the proposed DCT coefficient analysis methods are efficient and practical in terms of classification accuracy. In our experiment, the proposed approach yields a false positive rate of 0.32% and a true positive rate of 96.1% in terms of erroneous JPEG fragment classification.     

JPEG-compliant perceptual coding for a grayscale image printingpipeline

We describe a procedure by which Joint Photographic Experts Group (JPEG) compression may be customized for gray-scale images that are to be compressed before they are scaled, halftoned, and printed. Our technique maintains 100% compatibility with the JPEG standard, and is applicable with all scaling and halftoning methods. The JPEG quantization table is designed using frequency-domain characteristics of the scaling and halftoning operations, as well as the frequency sensitivity of the human visual system. In addition, the Huffman tables are optimized for low-rate coding. Compression artifacts are significantly reduced because they are masked by the halftoning patterns, and pushed into frequency bands where the eye is less sensitive. We describe how the frequency-domain effects of scaling and halftoning may be measured, and how to account for those effects in an iterative design procedure for the JPEG quantization table. We also present experimental results suggesting that the customized JPEG encoder typically maintains "near visually lossless" image quality at rates below 0.5 b/pixel (with reference to the number of pixels in the original image) when it is used with bilinear interpolation and either error diffusion or ordered dithering. Based on these results, we believe that in terms of the achieved bit rate, the performance of our encoder is typically at least 20% better than that of a JPEG encoder using the suggested baseline tables.     

Compressing image-based relighting data using eigenanalysis and wavelets

In image-based relighting (IBL) a tremendous number of reference images are needed to synthesise a high-quality novel image. This collection of reference images is referred as an IBL data set. An effective compression method for IBL data makes the IBL technique more practical. Within an IBL data set, there is a strong correlation among different reference images. In conventional eigen-based image compression methods, the principal component analysis (PCA) process is used for exploiting the correlation within a single image. Such an approach is not suitable for handling IBL data. The authors present an eigenimage-based method for compressing IBL data. The method exploits the correlation among reference images. Since there is a huge number of images and pixel values, the cascade recursive least square (CRLS) network based PCA is used to extract eigenimages. Afterwards, the wavelet approach is used for compressing those eigenimages. Simulation results demonstrate that this approach is much superior to that of compressing each reference image with JPEG and JPEG2000.     

Detection of visual attention regions in images using robust subspace analysis

In this paper, we describe a new framework to extract visual attention regions in images using robust subspace estimation and analysis techniques. We use simple features like hue and intensity endowed with scale adaptivity in order to represent smooth and textured areas in an image. A polar transformation maps homogeneity in the features into a linear subspace that also encodes spatial information of a region. A new subspace estimation algorithm based on the Generalized Principal Component Analysis (GPCA) is proposed to estimate multiple linear subspaces. Sensitivity to outliers is achieved by weighted least squares estimate of the subspaces in which weights calculated from the distribution of K nearest neighbors are assigned to data points. Iterative refinement of the weights is proposed to handle the issue of estimation bias when the number of data points in each subspace is very different. A new region attention measure is defined to calculate the visual attention of each region by considering both feature contrast and spatial geometric properties of the regions. Compared with existing visual attention detection methods, the proposed method directly measures global visual attention at the region level as opposed to pixel level.     

一种新型径向基函数神经网络的非线性系统逼近

讨论了一种新的、正弦型径向基函数(SRBF)神经网络，并用来逼近n堆连续函数。该SRBF所采用的n堆正弦型的基函数是光滑的,并且是致密的。该SRBF网络的权因子是输入的低阶多项式函数。本文给出的一种简单计算程序，显著地降低了网络训练和计算时间。并且由于SRBF的基函数可以非均匀的量化格点为中心。因而降低了网络所需存储的样本数，网络的输出及其一阶导数都是连续的。对于非线性系统。该SRBF网络在系统定义城内的逼近是精确的。并且在存储参数的个数上是最优的。通过实例仿真，证明该方法步骤简单,训练速度快，精度也很理想。     

A new image quality assessment method to detect and measure strength of blocking artifacts

Block based transform coding is one of the most popular techniques for image and video compression. However it suffers from several visual quality degradation factors, most notably from blocking artifacts. The subjective picture quality degradation caused by blocking artifacts, in general, does not agree well with the popular objective quality measure such as PSNR.A new image quality assessment method that detects and measures strength of blocking artifacts for block based transform coded images is proposed. In order to characterize the blocking artifacts, we utilize two observations: if blocking artifacts occur on the block boundary, the pixel value changes abruptly across the boundary and the same pixel values usually span along the entire length of the boundary. The proposed method operates only on a single block boundary to detect blocking artifacts. When a boundary is classified as having blocking artifacts, corresponding blocking artifact strength is also computed. Average values of those blocking artifact strengths are converted into a single number representing the subjective image quality. Experiments on various JPEG compressed images with various bit rates demonstrated that the proposed blocking artifacts measuring value matches well with the subjective image quality judged by human observers.     

Arithmetic coding for image compression with adaptive weight-context classification

In this paper, a new binary arithmetic coding strategy with adaptive-weight context classification is introduced to solve the context dilution and context quantization problems for bitplane coding. In our method, the weight, obtained using a regressive–prediction algorithm, represents the degree of importance of the current coefficient/block in the wavelet transform domain. Regarding the weights as contexts, the coder reduces the context number by classifying the weights using the Lloyd–Max algorithm, such that high-order is approximated as low-order context arithmetic coding. The experimental results show that our method effectively improves the arithmetic coding performance and outperforms the compression performances of SPECK, SPIHT and JPEG2000.     

Movie denoising by average of warped lines.

Here, we present an efficient method for movie denoising that does not require any motion estimation. The method is based on the well-known fact that averaging several realizations of a random variable reduces the variance. For each pixel to be denoised, we look for close similar samples along the level surface passing through it. With these similar samples, we estimate the denoised pixel. The method to find close similar samples is done via warping lines in spatiotemporal neighborhoods. For that end, we present an algorithm based on a method for epipolar line matching in stereo pairs which has per-line complexity O (N), where N is the number of columns in the image. In this way, when applied to the image sequence, our algorithm is computationally efficient, having a complexity of the order of the total number of pixels. Furthermore, we show that the presented method is unsupervised and is adapted to denoise image sequences with an additive white noise while respecting the visual details on the movie frames. We have also experimented with other types of noise with satisfactory results.     

混合式光场相机在聚焦与非聚焦模式下的超分辨重聚焦仿真

光场描述了光在自由空间传播的全四维信息,光场相机可用来获得光场图像。在传统的光场相机中,最终获得图像的空间分辨率受限于微透镜阵列中透镜的个数。聚焦型光场相机相较于传统光场相机能够获得更高的空间分辨率,但是以牺牲其角度分辨率作为代价。在Zemax中建立了传统光场相机与将聚焦光场相机的成像模型,仿真获得了两种光场相机的光场图像,分析了两种不同类型光场相机采样模式的区别。提出将可变焦液体透镜阵列放置在光场相机中,可以同时获得聚焦和非聚焦两种模式下的光场图像。根据记录的光场信息,讨论了相应的重聚焦方法,计算仿真了在不同景深下的重聚焦图像,并提出了一种基于图像融合和超分辨率重构的方法来提高重聚焦图像的分辨率,最终在相同的景深范围内获得了3倍于传统光场相机分辨率的重聚焦图像。     

Movie Denoising by Average of Warped Lines

Here, we present an efficient method for movie denoising that does not require any motion estimation. The method is based on the well-known fact that averaging several realizations of a random variable reduces the variance. For each pixel to be denoised, we look for close similar samples along the level surface passing through it. With these similar samples, we estimate the denoised pixel. The method to find close similar samples is done via warping lines in spatiotemporal neighborhoods. For that end, we present an algorithm based on a method for epipolar line matching in stereo pairs which has per-line complexity O(N) , where is the number of columns in the image. In this way, when applied to the image sequence, our algorithm is computationally efficient, having a complexity of the order of the total number of pixels. Furthermore, we show that the presented method is unsupervised and is adapted to denoise image sequences with an additive white noise while respecting the visual details on the movie frames. We have also experimented with other types of noise with satisfactory results.     

机器视觉中的微分测距方法研究

从全光函数的角度分析了利用全光微分即全光函数对各函数变量偏微分之间的关系来估计视场中物体距离,并给出了全光微分方法在机器视觉中的几个具体实施范例。结合估计算法简单地比较了几种方法的优劣。     

Non-local spatial redundancy reduction for bottom-up saliency estimation

In this paper we present a redundancy reduction based approach for computational bottom-up visual saliency estimation. In contrast to conventional methods, our approach determines the saliency by filtering out redundant contents instead of measuring their significance. To analyze the redundancy of self-repeating spatial structures, we propose a non-local self-similarity based procedure. The result redundancy coefficient is used to compensate the Shannon entropy, which is based on statistics of pixel intensities, to generate the bottom-up saliency map of the visual input. Experimental results on three publicly available databases demonstrate that the proposed model is highly consistent with the subjective visual attention.     

MATIA: a programmable 80 /spl mu/W/frame CMOS block matrix transform imager architecture

In this paper, we introduce our CMOS block MAtrix Transform Imager Architecture (MATIA). This imager is capable of performing programmable matrix operations on an image. The imager architecture is both modular and programmable. The pixel used in this architecture performs matrix multiplication while maintaining a high fill factor (46%), comparable to active pixel sensors. Floating gates are used to store the arbitrary matrix coefficients on-chip. The chip operates in the subthreshold domain and thus has low power consumption (80 /spl mu/W/frame). We present data for different convolutions and block transforms that were implemented using this architecture, and also present data from baseline JPEG and motion JPEG systems which we have implemented using MATIA.     

Preserving radiometric resolution in remotely sensed data withlossy compression

Recent advances in imaging technology make it possible to obtain remotely sensed imagery data of the Earth at high spatial, spectral, and radiometric resolutions. The rate at which the data is collected from these satellites can far exceed the channel capacity of the data downlink. Reducing the data rate to within the channel capacity can often require painful trade-offs in which certain scientific returns are sacrificed for the sake of others. The authors focus on the case where radiometric resolution is sacrificed by dropping a specified number of lower order bits (LOB) from each data pixel. To limit the number LOBs dropped, they also compress the remaining bits using lossless compression. They call this approach “truncation followed by lossless compression” or TLLC. They then demonstrate the suboptimality of this TLLC approach by comparing it with the direct application of a more effective lossy compression technique based on the JPEG algorithm. This comparison demonstrates that, for a given channel rate, the method based on JPEG lossy compression better preserves radiometric resolution than does TLLC     

JPEG2000实时截断码率控制新算法及其VLSI结构设计

提出一种实时编码实时截断的码率控制算法.它根据已分解的小波子带内码块有效位平面数来预测未分解的小波子带内码块有效位平面数,并根据编码通道数和小波/量化权系数为当前编码码块分配码率.并提出一种JPEG2000编码实时截断,两级码率控制的编码体系结构.第一级采用本文提出的算法实时截断码流和编码通道.第二级在低码率下采用JPEG2000标准的PCRD优化算法搜索精确的分层截断点.在最优分层截断之前多数码流和编码通道被预先截断,存储器损耗小,实时性高.低码率下,图像质量跟JPEG2000标准一致.     

用于亚公里级三维成像的全光相机主物镜设计

The plenoptic camera can refocus after imaging, and obtain the position and direction information of the target at the same time with one exposure. Compared with the active distance measurement method and the traditional passive distance measurement method, the depth measurement method based on the plenoptic camera has the advantages of being difficult to detect and easy to calibrate. The plenoptic camera 3D imaging technology is a computational imaging technology that integrates the front-end optical system and the back-end information processing. The current research works mainly focus on the back-end information processing algorithm et al. There are few reports on the research of the front-end optical system. Therefore, the design of the front-end optical system was researched. Firstly, a calculation model was established for the depth resolution of a plenoptic camera based on multi-eye vision and the influence of optical system performance parameters was analyzed such as focal length and F-number on the object depth resolution. Secondly, the influence of factors was analyzed such as the blocking ratio of the two-mirror optical system and the magnification of the secondary mirror on the system parameters. Finally, a plenoptic camera main objective optical system for sub-kilometer-scale 3D imaging was designed comprehensively considering the design, processing, assembly, and ranging performance. The focal length of the system is 500 mm, the total length of the system is less than 163 mm, the telephoto ratio is less than 1/3, and the working temperature range is ?40 -70 ℃. The full field of view MTF in 80 lp/mm is better than 0.3 at different temperatures. If the plenoptic camera uses this objective and a sub-pixel recognition accuracy algorithm of 1/8 pixel, a depth resolution of less than 5 m can be obtained at 0.5 km.