Dictionary learning based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient reconstruction by leveraging more realistic signal models that go beyond simple sparsity is still an open challenge. In this paper, we propose a novel “undersampled” correlation noise model to describe compressively sampled video signals, and present a maximum-likelihood dictionary learning based reconstruction algorithm for DCVS, in which both the correlation and sparsity constraints are included in a new probabilistic model. Moreover, the signal recovery in our algorithm is performed during the process of dictionary learning, instead of being employed as an independent task. Experimental results show that our proposal compares favorably with other existing methods, with 0.1–3.5 dB improvements in the average PSNR, and a 2–9 dB gain for non-key frames when key frames are subsampled at an increased rate.     

Correlation estimation from compressed images

This paper addresses the problem of correlation estimation in sets of compressed images. We consider a framework where the images are represented under the form of linear measurements due to low complexity sensing or security requirements. We assume that the images are correlated through the displacement of visual objects due to motion or viewpoint change and the correlation is effectively represented by optical flow or motion field models. The correlation is estimated in the compressed domain by jointly processing the linear measurements. We first show that the correlated images can be efficiently related using a linear operator. Using this linear relationship we then describe the dependencies between images in the compressed domain. We further cast a regularized optimization problem where the correlation is estimated in order to satisfy both data consistency and motion smoothness objectives with a Graph Cut algorithm. We analyze in detail the correlation estimation performance and quantify the penalty due to image compression. Extensive experiments in stereo and video imaging applications show that our novel solution stays competitive with methods that implement complex image reconstruction steps prior to correlation estimation. We finally use the estimated correlation in a novel joint image reconstruction scheme that is based on an optimization problem with sparsity priors on the reconstructed images. Additional experiments show that our correlation estimation algorithm leads to an effective reconstruction of pairs of images in distributed image coding schemes that outperform independent reconstruction algorithms by 2–4 dB.     

Correlation based universal image/video coding loss recovery

Coding artifacts are annoying in highly compressed signals. Most of the existing artifact reduction methods are designed for one specific type of artifacts, codecs, and bitrates, which are complex and exclusive for one type of artifact reduction. Since both the compressed image/video and the coding error contain information of the original signal, they are highly correlated. Therefore, we try to recover some lost data based on the correlation between the compressed signal and the coding error, and introduce a novel and universal artifact reduction method. Firstly, according to the spatial correlation among pixels, a pixel-adaptive anisotropic filter is designed to reconstruct the distorted signal. Next, a globally optimal filter is designed to further recover the coding loss. Experimental results demonstrate that within an extensive range of bitrates, the proposed method achieves about 0.8 dB, 0.45 dB, 0.3 dB, and 0.2 dB on average of PSNR improvement for JPEG, MPEG4, H.264/AVC, and HEVC compressed signals, respectively.     

Video compressed sensing reconstruction based on structural group sparsity and successive approximation estimation model

The existing video compressed sensing (CS) algorithms for inconsistent sampling ignore the joint correlations of video signals in space and time, and their reconstruction quality and speed need further improvement. To balance reconstruction quality with computational complexity, we introduce a structural group sparsity model for use in the initial reconstruction phase and propose a weight-based group sparse optimization algorithm acting in joint domains. Then, a coarse-to-fine optical flow estimation model with successive approximation is introduced for use in the interframe prediction stage to recover non-key frames through alternating optical flow estimation and residual sparse reconstruction. Experimental results show that, compared with the existing algorithms, the proposed algorithm achieves a peak signal-to-noise ratio gain of 1–3 dB and a multi-scale structural similarity gain of 0.01–0.03 at a low time complexity, and the reconstructed frames not only have good edge contours but also retain textural details.     

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner–Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder.One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion.In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5 dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.     

Saliency-guided compressive sensing approach to efficient laser range measurement

The acquisition of laser range measurements can be a time consuming process for situations where high spatial resolution is required. As such, optimizing the acquisition mechanism is of high importance for many range measurement applications. Acquiring such data through a dynamically small subset of measurement locations can address this problem. In such a case, the measured information can be regarded as incomplete, which necessitates the application of special reconstruction tools to recover the original data set. The reconstruction can be performed based on the concept of sparse signal representation. Recovering signals and images from their sub-Nyquist measurements forms the core idea of compressive sensing (CS). A new saliency-guided CS-based algorithm for improving the reconstruction of range image from sparse laser range measurements has been developed. This system samples the object of interest through an optimized probability density function derived based on saliency rather than a uniform random distribution. Particularly, we demonstrate a saliency-guided sampling method for simultaneously sensing and coding range image, which requires less than half the samples needed by conventional CS while maintaining the same reconstruction performance, or alternatively reconstruct range image using the same number of samples as conventional CS with a 16 dB improvement in signal-to-noise ratio. For example, to achieve a reconstruction SNR of 30 dB, the saliency-guided approach required 30% of the samples in comparison to the standard CS approach that required 90% of the samples in order to achieve similar performance.     

Free-viewpoint depth image based rendering

In 3D TV research, one approach is to employ multiple cameras for creating a 3D multi-view signal with the aim to make interactive free-viewpoint selection possible in 3D TV media. This paper explores a new rendering algorithm that enables to compute a free-viewpoint between two reference views from existing cameras. A unique property is that we perform forward warping for both texture and depth simultaneously. Advantages of our rendering are manyfold. First, resampling artifacts are filled in by inverse warping. Second, disocclusions are processed while omitting warping of edges at high discontinuities. Third, our disocclusion inpainting approach explicitly uses depth information. We obtain an average PSNR gain of 3 dB and 4.5 dB for the ‘Breakdancers’ and ‘Ballet’ sequences, respectively, compared recently published results. Moreover, experiments are performed using compressed video from surrounding cameras. The overall system quality is dominated by rendering quality and not by coding.     

Frame-wise detection of relocated I-frames in double compressed H.264 videos based on convolutional neural network

Relocated I-frames are a key type of abnormal inter-coded frame in double compressed videos with shifted GOP structures. In this work, a frame-wise detection method of relocated I-frame is proposed based on convolutional neural network (CNN). The proposed detection framework contains a novel network architecture, which initializes with a preprocessing layer and is followed by a well-designed CNN. In the preprocessing layer, the high-frequency component extraction operation is applied to eliminate the influence of diverse video contents. To mitigate overfitting, several advanced structures, such as 1 × 1 convolutional filter and the global average-pooling layer, are carefully introduced in the design of the CNN architecture. Public available YUV sequences are collected to construct a dataset of double compressed videos with different coding parameters. According to the experiments, the proposed framework can achieve a more promising performance of relocated I-frame detection than a well-known CNN structure (AlexNet) and the method based on average prediction residual.     

Just noticeable distortion model and its applications in video coding

We explore a new perceptually-adaptive video coding (PVC) scheme for hybrid video compression, in order to achieve better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color video is first devised in the image domain. How to efficiently integrate masking effects together is a key issue of JND modelling. We integrate spatial masking factors with the nonlinear additivity model for masking (NAMM). The JND estimator applies to all color components and accounts for the compound impact of luminance masking, texture masking and temporal masking. Extensive subjective viewing confirms that it is capable of determining a more accurate visibility threshold that is close to the actual JND bound in human eyes. Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation and residue filtering process. The scheme works with any prevalent video coding standards and various motion estimation strategies. To demonstrate the effectiveness of the proposed scheme, it has been implemented in the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in motion estimation efficiency, 0.6 dB in average peak signal-to perceptual-noise ratio (PSPNR) and most remarkably, 0.17 dB in the objective coding quality measure (PSNR) on average. Theoretical explanation is presented for the improvement on the objective coding quality measure. With the JND-based motion estimation and residue filtering process, hybrid video encoding can be more efficient and the use of bits is optimized for visual quality.     

New rate-distortion modeling and efficient rate control for H.264/AVC video coding

Rate control (RC) is crucial in controlling compression bit rates and qualities for networked video applications. In this paper, we propose a new rate-distortion (R-D) model and an efficient rate control scheme for H.264/AVC video coding, which elegantly resolve the inter-dependency problem between rate-distortion optimization and rate control by eliminating the need of coding complexity prediction for an inter-frame. The objective is to achieve accurate bit rate, obtain optimal video quality while reducing quality variations and simultaneously handling buffer fullness effectively. The proposed algorithm encapsulates a number of new features, including a coding complexity measure for intra-frames, a rate-distortion model, an accurate quantization parameter (QP) estimation for intra-frames, an incremental quantization parameter calculation method for inter-frames, a proportional+integral+derivative (PID) buffer controller, and an intelligent bit-allocation-balancing technique. Our experimental results demonstrate that the proposed scheme outperforms the JVT-G012 solution by providing accurate rate regulation, effectively reducing frame skipping, and finally improving coding quality by up to 1.80 dB.     

A 24-dB Ku-band low-power linearized 90-nm amplifier with a built-in output buffer

In this paper, we present a 90-nm high gain (24 dB) linearized CMOS amplifier suitable for applications requiring high degree of port isolation in the K_u-band (13.2–15.4 GHz). The two-stage design is composed of a low-noise common-gate stage and a gain-boosting cascode block with an integrated output buffer for measurement. Optimization of input stage and load-port buffer parameters improves the front-end's linear coverage, port return-loss, and overall gain without burdening its power demand and noise contribution. With low gate bias voltages (0.65–1.2 V) and an active current source, <?10 dB port reflection loss and 3.25–3.41 dB NF are achieved over the bandwidth. The input reflection loss of the overall amplifier lies between ?35 and ?10 dB and the circuit demonstrates a peak forward gain of 24 dB at 14.2 GHz. The output buffer improves the amplifier's forward gain by ～9 dB and pushes down the minimum output return loss to ?22.5 dB while raising the front-end NF by only 0.05 dB. The effect of layout parasites is considered in detail in the 90-nm process models for accurate RF analysis. Monte Carlo simulation predicts 9% and 8% variation in gain and noise figures resulting from a 10% mismatch in process. The K_u-band amplifier including the buffer block consumes 7.69 mA from a 1.2-V supply. The proposed circuit techniques achieve superior small signal gain, GHz-per-milliwatt, and range of linearity when compared with simulated results of reported microwave amplifiers.     

SIFT-flow-based color correction for multi-view video

During the multi-view video acquisition, color variation across the views tends to be incurred due to different camera positions, orientations, and local lighting conditions. Such color variation will inevitably deteriorate the performance of the follow-up multi-view video processing, such as multi-view video coding (MVC). To address this problem, an effective color correction algorithm, called the SIFT flow-based color correction (SFCC), is proposed in this paper. First, the SIFT-flow technique is used to establish point-to-point correspondences across all the views of the multi-view video. The average color is then computed based on those identified common corresponding points and used as the reference color. By minimizing the energy of the difference yielded between the color of those identified common corresponding points in each view with respect to the reference color, the color correction matrix for each view can be obtained and used to correct its color. Experimental results have shown that the proposed SFCC algorithm is able to effectively eliminate the color variation inherited in multi-view video. By further exploiting the developed SFCC algorithm as a pre-processing for the MVC, extensive simulation results have shown that the coding efficiency of the color-corrected multi-view video can be greatly improved (on average, 0.85 dB, 1.27 dB and 1.63 dB gain for Y, U, and V components, respectively), compared with that of the original multi-view video without color correction.     

Performance analysis,parameter selection and extensions to H.264/AVC FRExt for high resolution video coding

H.264/AVC FRExt (Fidelity Range Extensions) and Motion JPEG 2000 are the current respective inter-frame and intra-frame coding standards for high resolution (HR) (e.g., 4096 × 2160) visual signals. It is commonly believed that an inter-frame method could achieve higher coding efficiency compared with an intra-frame one, due to the exploitation of video temporal redundancy. However, Motion JPEG 2000 has been selected as the digital cinema compression standard, and some existing work has demonstrated that JPEG 2000 is more suitable at HR situations. In this paper, we compare the rate–distortion (R–D) performance of these two different schemes and give more insight from both theoretical and experimental point of view. We derive an entropy-based R–D model to analyze the test results and the impact of residual entropy and quantization for inter-frame coding. Several extensions are introduced into H.264/AVC FRExt for HR video content for better performance. Experimental results show that these extensions lead to significantly higher coding efficiency and make our extended version more suitable for HR video coding     

A high current gain gate-controlled lateral bipolar junction transistor with 90 nm CMOS technology for future RF SoC applications

A CMOS-compatible gate-controlled lateral BJT (GC-LBJT) was prepared with a conventional 90 nm CMOS technology for radio frequency system-on-chip (RF SoC) applications. The emitter injection efficiency and the doping profile in P-well were optimized by properly controlling source, drain, and well implants. Consequently, the GC-LBJT with a gate length of 0.15 μm can achieve a current gain over 2000 and 17/19 GHz for the f_T/f_max, respectively, which are 1000%, 200%, and 60% improvements in current gain, f_T and f_max, respectively as compared to the LBJT reported previously.     

Rate-distortion optimization of scalable video codecs

In this paper joint optimization of layers in the layered video coding is investigated. Through theoretical analysis and simulations, it is shown that, due to higher interactions between the layers in a SNR scalable codec, this type of layering technique benefits most from joint optimization of the layers. A method for joint optimization is then proposed, and its compression efficiency is contrasted against the separate optimization and an optimized single layer coder. It is shown that, in joint optimization of SNR scalable coders when the quantization step size of the enhancement layer is larger than half the step size of the base layer, an additional improvement is gained by not sending the enhancement zero valued quantized coefficients, provided they are quantized at the base-layer. This will result in a non-standard bitstream syntax and as an alternative for standard syntax, one may skip the inter coded enhancement macroblocks. Through extensive tests it is shown that while separate optimization of SNR coders is inferior to single layer coder by more than 2 dB, with joint optimization this gap is reduced to 0.3–0.5 dB. We have shown that through joint optimization quality of the base layer video is also improved over the separate optimization. It is also shown that spatial scalability like SNR scalability does benefit from joint optimization, though not being able to exploit the relation between the quantizer step sizes. The amount of improvement depends on the interpolation artifacts of upsampled base-layer and the residual quantization distortion of this layer. Hence, the degree of improvement depends on image contents as well as the bit rate budget. Simulation results show that joint optimization of spatial scalable coders is about 0.5–1 dB inferior to the single layer optimized coder, where its separate optimization counterpart like SNR scalability is more than 2 dB worse.     

p-Layer bandgap engineering for high efficiency thin film silicon solar cells

Spectroscopic ellipsometry (SE), high resolution transmission electron microscopy (HRTEM), atomic force microscopy (AFM) and optical transmittance measurements were used to study and establish a correlation between the open-circuit voltage (V_oc) of solar cells and the p-layer optical band gap (E_p). It is found that the ellipsometry measurement can be used as an inline non-destructive diagnostic tool for p-layer deposition in commercial operation. The analysis of ellipsometric spectra, together with the optical transmittance data, shows that the best p-layer appears to be very fine nanocrystallites with an E_p 1.95 eV. HRTEM measurements reveal that the best p-layer is composed of nanocrystallites ~9 nm in size. It is also found that the p-layer exhibits very good transmittance, as high as ~91.6% at ~650 nm. These results have guided us to achieve high V_oc value 1.03 V for thin film silicon based single junction solar cell.     

An automatic region-based video sequence codec based on fractal compression

In order to improve the performance of fractal video coding, we explore a novel fractal video sequences codec with automatic region-based functionality. To increase the quality of decoding image, intra frame coding, deblocking loop filter and sub-pixel block matching are applied to the codec. An efficient searching algorithm is used to increase the compression ratio and encoding speed. Automatic region-based fractal video sequences coding reduces coding stream greatly. Experimental results indicate that the proposed algorithm is more robust, and provides much less encoding time and bitrate while maintaining the quality of decompression image than the conventional CPM/NCIM method and other related references. We compare the proposed algorithm with three algorithms in Refs. [24], [25], [26], and the results of all these four algorithms are compared with H.264. The bitrate of the proposed algorithm is decreased by 0.11% and the other algorithms are increased by 4.29%, 6.85% and 11.62%, respectively. The average PSNR degradations of the four algorithms are 0.71 dB, 0.48 dB, 0.48 dB and 0.75 dB. So the bitrate of the proposed algorithm is decreased and the other algorithms are increased. At the meantime the compression time is reduced greatly, about 79.19% on average. The results indicate that, on average, the proposed automatic region-based fractal video sequences coding system can save compression time 48.97% and bitrate 52.02% with some image quality degradation in comparison with H.264, since they are all above 32 dB and the human eyes are insensitive to the differences.     

Microwave properties and applications of Y–Ba–Cu–O thin films grown on various substrates

Microwave properties of YBa₂Cu₃O_7-δ (YBCO) films grown on (100) LaAlO₃ (LAO), (110) NdGaO₃ (NGO) and (001) SrLaAlO₄ (SLAO) substrates were studied in the form of a microstrip ring resonator at temperatures above 20 K. The YBCO resonator on a SLAO substrate showed microwave properties better than or comparable to other YBCO resonators on LAO substrates. For the YBCO resonators on LAO and SLAO substrates, both Q_U and f₀ appeared to decrease as the temperature was raised. Meanwhile the resonator on a NGO substrate showed different behaviors with Q_U showing a peak at ∼70 K, which are attributed to the unique temperature dependence of the loss tangent of the NGO substrate. An X-band oscillator with a YBCO ring resonator coupled to the circuit was prepared and its properties were investigated at low temperatures. The frequency of the oscillator signal appeared to change from 7.925 GHz at 30 K to 7.878 GHz at 77 K, which was mostly attributed to the change in f₀ of the YBCO ring resonator. The signal power appeared to be more than 4.5 mW at 30 K and 2.1 mW at 77 K, respectively. At 55 K, the frequency of the oscillator signal was 7.917 GHz with the 3 dB-linewidth of 450 Hz.     

Response time and mechanism of Pd modified TiO2 gas sensor

In this work, gas response properties of Pd modified TiO₂ sensing films are discussed when exposed to H₂ and O₂. TiO₂ films are surface modified in PdCl₂-containing solution by the dipping method and treated for different treatment times to get different surface states. X-ray diffraction (XRD), X-ray photoelectron spectroscopy (XPS), scanning electron microscopy (SEM) and Kröger–Vink defect theory are used to characterize the sensing films. The gas response properties indicate that the sensor response time which related to the rate of change of sensor resistance is affected by the activation energy (E). In particular, the sensor treated at 900 °C for 2 h exhibits a response time of about 20–240 ms when exposed to H₂ and 40–130 ms when exposed to O₂ at 500–800 °C.     

Fuzzy quantization based bit transform for low bit-resolution motion estimation

This study proposes a novel fuzzy quantization based bit transform for low bit-resolution motion estimation. We formalize the procedure of bit resolution reduction by two successive steps, namely interval partitioning and interval mapping. The former is a many-to-one mapping which determines motion estimation performance, while the latter is a one-to-one mapping. To gain a reasonable interval partitioning, we propose a non-uniform quantization method to compute coarse thresholds. They are then refined by using a membership function to solve the mismatch of pixel values near threshold caused by camera noise, coding distortion, etc. Afterwards, we discuss that the sum of absolute difference (SAD) is one of the fast matching metrics suitable for low bit-resolution motion estimation in the sense of mean squared errors. A fuzzy quantization based low bit-resolution motion estimation algorithm is consequently proposed. Our algorithm not only can be directly employed in video codecs, but also be applied to other fast or complexity scalable motion estimation algorithms. Extensive experimental results show that the proposed algorithm can always achieve good motion estimation performances for video sequences with various characteristics. Compared with one-bit transform, multi-thresholding two-bit transform, and adaptive quantization based two-bit transform, our bit transform separately gains 0.98 dB, 0.42 dB, and 0.24 dB improvement in terms of average peak signal-to-noise ratio, with less computational cost as well.