Maximization of Nonlinear Autocorrelation for Blind Source Separation of Non-stationary Complex Signals

Blind source separation of complex-valued signals has been a vital issue especially in the field of digital communication signal processing. This paper proposes a novel method based on nonlinear autocorrelation to solve the problem. Relying on the temporal structure with nonlinear autocorrelation of the signals, the method has a potential capability of extracting non-stationary complex sources with Gaussian or non-Gaussian distribution. Most traditional methods would fail in separating this kind of sources. We also analyze the stability conditions of the method in theory. Numerical simulations on artificial complex Gaussian data and orthogonal frequency division multiplexing sources corroborate the validity and efficiency of the proposed method. Moreover, with respect to classical methods, including cumulant-based approach using the non-stationarity of variance and complexity pursuit, our method offers equally good results with lower computational cost and better robustness. Finally, experiments for the separation of real communication signals illustrate that our method has good prospects in real-world applications.     

An iterative method using conditional second-order statistics applied to the blind source separation problem

This paper is concerned with the problem of blind separation of an instantaneous mixture of sources (BSS), which has been addressed in many ways. When power spectral densities of the sources are different, methods using second-order statistics are sufficient to solve this problem. Otherwise, these methods fail and others (higher order statistics, etc.) must be used. In this paper, we propose an iterative method to process the case of sources with the same power spectral density. This method is based on an evaluation of conditional first and second-order statistics only. Restrictions on characteristics of sources are given to reach a solution, and proofs of convergence of the algorithm are provided for particular cases of probability density functions. Robustness of this algorithm with respect to the number of sources is shown through computer simulations. A particular case of sources that have a probability density function with unbounded domain of definition is described; here, the algorithm does not lead directly to a separation state but to an a priori known mixture state. Finally, prospects of links with contrast functions are mentioned, with a possible generalization of them based on results obtained with particular sources.     

联合非高斯性和非线性自相关的有噪盲源分离算法

现有的盲源分离算法往往利用信号某一方面的统计特性来分离信号,例如:利用信号的非高斯特性,或者利用信号的时序特性。在实际应用中,信号往往是具有这两种特性信号的混合,采用信号某一方面的特性往往不能够成功的分离出信号。现有的盲源分离算法往往不考虑噪声的影响,但在实际应用中,噪声的影响是不可避免的。当源信号具有非高斯性和非线性自相关特性时,提出了联合非高斯性和非线性自相关特性的有噪盲源分离算法。计算机仿真表明了提出算法的有效性,和现有的基于非高斯性和非线性自相关特性的有噪盲源分离算法相比,提出算法具有更好的信号分离性能。      

Subspace-Based Channel Shortening for the Blind Separation of Convolutive Mixtures

A novel subspace-based channel shortening procedure is proposed based on the structure of the delayed autocorrelation matrices of the observation process. This purely second-order approach applies to overdetermined multiple-input multiple-output (MIMO) channels with independent, white sources. The channel may be sparse, and its length is assumed to be unknown. Through successive deflations, the problem can be transformed into an instantaneous blind source separation (BSS) problem which is simpler to solve using, for example, independent component analysis (ICA) techniques. The algorithm is computationally fast although it requires large input datasets. Such data can be acquired either through large numbers of sensors or by using increased data sampling rate. When not enough data are available, the method can still be used for reducing the channel length thus simplifying the problem for subsequent treatment.     

Diffusion approximation modeling for Markov modulated burstytraffic and its applications to bandwidth allocation in ATMnetworks

We consider a statistical multiplexer model, in which each of the K sources is a Markov modulated rate process (MMRP). This formulation allows a more general source model than the well studied “on-off” source model in characterizing variable bit rate (VBR) sources such as compressed video. In our model we allow an arbitrary distribution for the duration of each of the M states (or levels) that the source can take on. We formulate Markov modulated sources as a closed queueing network with M infinite-server nodes. By extending our earlier results we introduce an M-dimensional diffusion process to approximate the aggregate traffic of such Markov modulated sources. Under a set of reasonable assumptions we then show that this diffusion process can be expressed as an M-dimensional Ornstein-Uhlenbeck (O-U) process. The queueing behavior of the buffer content is analyzed by applying a diffusion process approximation to the aggregate arrival process. We show some numerical examples which illustrate typical sample paths, and autocorrelation functions of the aggregate traffic and its diffusion process representation. Simulation results validate our proposed approximation model, showing good fits for distributions and autocorrelation functions of the aggregate rate process and the asymptotic queueing behavior. We also discuss how the analytical formulas derived from the diffusion approximation can be applied to compute the equivalent bandwidth for real-time call admission control, and how the model can be modified to characterize traffic sources with long-range dependence     

A Markov model for blind image separation by a mean-field EM algorithm.

This paper deals with blind separation of images from noisy linear mixtures with unknown coefficients, formulated as a Bayesian estimation problem. This is a flexible framework, where any kind of prior knowledge about the source images and the mixing matrix can be accounted for. In particular, we describe local correlation within the individual images through the use of Markov random field (MRF) image models. These are naturally suited to express the joint pdf of the sources in a factorized form, so that the statistical independence requirements of most independent component analysis approaches to blind source separation are retained. Our model also includes edge variables to preserve intensity discontinuities. MRF models have been proved to be very efficient in many visual reconstruction problems, such as blind image restoration, and allow separation and edge detection to be performed simultaneously. We propose an expectation-maximization algorithm with the mean field approximation to derive a procedure for estimating the mixing matrix, the sources, and their edge maps. We tested this procedure on both synthetic and real images, in the fully blind case (i.e., no prior information on mixing is exploited) and found that a source model accounting for local autocorrelation is able to increase robustness against noise, even space variant. Furthermore, when the model closely fits the source characteristics, independence is no longer a strict requirement, and cross-correlated sources can be separated, as well.     

Covariance and time-scale methods for blind separation of delayed sources

An extension of the blind source separation technique based on the second-order blind identification (SOBI) approach is presented to separate mixtures of delayed sources. When the delay is small such that the first-order Taylor approximation holds, the delayed mixture is transformed as the mixture of the original sources and their derivatives. Two algorithms are proposed for the rotation step that recovers the extended source vector (original sources and the corresponding derivatives). The first approach is based on the odd symmetry of the derivative of the autocorrelation function; and the second method identifies the locations of single auto terms in the optimized time-scale plane. A simulation analysis was conducted to evaluate the performance of the proposed algorithms. The results showed that the proposed methods substantially improved the performance of SOBI and its extension in the time-scale plane when the sources presented delays in the mixtures. In addition, the proposed algorithms were applied representatively to experimental multichannel surface electromyographic signals to identify motor unit action potential trains from the interference signal. The performance of the proposed methods was superior to previous methods also in this representative application. In conclusion, extensions of the SOBI approach of source separation have been proposed for the case of sources being delayed in the mixtures. These techniques were proven superior to previous approaches.     

Tunable line spectral estimators based on state-covariance subspace analysis

Subspace methods for spectral analysis can be adapted to the case where state covariance of a linear filter replaces the traditional Toeplitz matrix formed out of a partial autocorrelation sequence of a time series. This observation forms the basis of a new framework for spectral analysis. The goal of this paper is to quantify potential advantages in working with state-covariance data instead of the autocorrelation sequence. To this end, we identify tradeoffs between resolution and robustness in spectral estimates and how these are affected by the filter dynamics. The approach leads to a novel tunable high-resolution frequency estimator.     

A Class of Sequential Blind Source Separation Method in Order Using Swarm Optimization Algorithm

We consider the problem of sequential, blind source separation in some specific order from a mixture of sub- and sup-Gaussian sources. Three methods of separation are developed, specifically, kurtosis maximization using (a) particle swarm optimization, (b) differential evolution, and (c) artificial bee colony algorithm, all of which produce the separation in decreasing order of the absolute kurtosis based on the maximization of the kurtosis cost function. The validity of the methods was confirmed through simulation. Moreover, compared with other conventional methods, the proposed method separated the various sources with greater accuracy. Finally, we performed a real-world experiment to separate electroencephalogram (EEG) signals from a super-determined mixture with Gaussian noise. Whereas the conventional methods separate simultaneously EEG signals of interest along with noise, the result of this example shows the proposed methods recover from the outset solely those EEG signals of interest. This feature will be of benefit in many practical applications.     

AEWMA control chart for monitoring variability sources of solder joints quality

A statistical approach for monitoring solder joint quality is presented in this paper. In semiconductor manufacturing, there are often multiple independent root causes (variability sources) that contribute to the overall observed variability in the measured profile. Each variability source may result in a distinct spatial pattern across some of the measured product characteristics. A combinational blind source separation method is proposed to recognize these patterns based on a high-order statistical analysis of inspection data. Visualization of the resulting patterns is shown to help illustrate the nature of their root causes. For the identified individual variability sources, we apply autocorrelation exponentially weighted moving average control charts to monitor the mean shifts by accommodating their autocorrelation and non-Gaussian distributions. The proposed control chart also facilitates online monitoring of solder joint quality by avoiding the sophisticated time-series modeling.     

Quality-of-Service Analysis for Statistical Multiplexing with Gaussian Distributed and Autoregressive Input

We investigate multiplexers in telecommunication systems with a workload process developing equivalent to that of a service system with semi-Markovian input, which includes fluid flow and time slotted systems. Discrete time methods are used to analyze their waiting time and loss rate.Our focus is on the performance evaluation of statistical multiplexing. Traffic flows are modeled by autoregressive processes producing autocorrelated and Gaussian distributed workload increases. The superposition of on-off voice sources approaches autoregressive processes and they also serve as a basic model for video traffic in an appropriate time scale, although video reveals a more complex autocorrelation structure. Performance results are obtained depending on only two parameters, which allow for clear conclusions about the statistical multiplexing gain with regard to bounds on loss rates as demanded in quality-of-service guarantees.     

Correlated Postfiltering and Mutual Information in Pseudoanechoic Model Based Blind Source Separation

In a recent publication the pseudoanechoic mixing model for closely spaced microphones was proposed and a blind audio sources separation algorithm based on this model was developed. This method uses frequency-domain independent component analysis to identify the mixing parameters. These parameters are used to synthesize the separation matrices, and then a time-frequency Wiener postfilter to improve the separation is applied. In this contribution, key aspects of the separation algorithm are optimized with two novel methods. A deeper analysis of the working principles of the Wiener postfilter is presented, which gives an insight in its reverberation reduction capabilities. Also a variation of this postfilter to improve the performance using the information of previous frames is introduced. The basic method uses a fixed central frequency bin for the estimation of the mixture parameters. In this contribution an automatic selection of the central bin, based in the information of the separability of the sources, is introduced. The improvements obtained through these methods are evaluated in an automatic speech recognition task and with the PESQ objective quality measure. The results show an increased robustness and stability of the proposed method, enhancing the separation quality and improving the speech recognition rate of an automatic speech recognition system.     

Stability analyses of information-theoretic blind separationalgorithms in the case where the sources are nonlinear processes

A basic approach to blind source separation is to define an index representing the statistical dependency among the output signals of the separator and minimize it with respect to the separator's parameters. The most natural index might be mutual information among the output signals of the separator. In the case of a convolutive mixture, however, since the signals must be treated as a time series, it becomes very complicated to concretely express the mutual information as a function of the parameters. To cope with this difficulty, in most of the conventional methods, the source signals are assumed to be independent identically distributed (i.i.d.) or linear. Based on this assumption, some simpler indices are defined, and their minimization is made by such an iterative calculation as the gradient method. In actual applications, however, the sources are often not linear processes. This paper discusses what will happen when those algorithms postulating the linearity of the sources are applied to the case of nonlinear sources. An analysis of local stability derives a couple of conditions guaranteeing that the separator stably tends toward a desired one with iteration. The obtained results reveal that those methods, which are based on the minimization of some indices related to the mutual information, do not work well when the sources signals are far from linear     

Application of the mutual information principle to spectral density estimation

The power spectrum of a stationary Gaussian random process is estimated when partial knowledge of the autocorrelation function is available {em a priori}. Particular attention is paid to the case when the {em a priori} knowledge is not precise, i.e., when there are errors in the measurements, perhaps due to the presence of noise. In the special case when the {em a priori} knowledge consists ofnpoints of the autocorrelation function, Burg's method of picking the spectrum which maximizes the entropy of the Gaussian process has been recently extended by Newman to account for a weighted average error in the estimates of the correlation function points. A new method is suggested here that uses the mutual information principle (MIP) of Tzannes and Noonan. The firstnpoints of the correlation function (obtained with errors) are used to derive an approximate spectrum by Burg's or any other method. This spectrum, as well as the error constraints involved, is then used to arrive at the underlying spectrum in the framework of the MIP approach.     

Novel Fast Computation Algorithm of the Second-Order Statistics for Autoregressive Moving-Average Processes

Very often, random signals are modeled as autoregressive moving-average (ARMA) processes in engineering and scientific applications. The acquisition of the second-order statistics or the calculation of the autocorrelation function for an ARMA process is very important in those applications. In this paper, we discuss two kinds of ARMA-autocorrelation computation approaches, namely the recursive approach and the direct approach. To overcome the problems of memory usage and computational burden, we design a new ARMA-autocorrelation calculation algorithm which belongs to the direct approach. Our novel algorithm originates from the partial fractional decomposition for the rational trigonometric functions and the associated definite integrals. For comparison, we also provide the theoretical analysis of computational complexity for our new scheme and the conventional recursion-based algorithm. From our studies, it can be shown that our new ARMA-autocorrelation calculation method is more efficient than the conventional algorithm when the underlying autocorrelation function is long or the corresponding power spectrum possesses the narrowband characteristics. On the other hand, the conventional algorithm is more efficient than our new algorithm when the model orders of the ARMA process are large and the underlying autocorrelation function is short (the corresponding power spectrum is wideband). Since our proposed new scheme involves the root-finding of a polynomial, the effect of the error-tolerance in solving polynomials on the induced complexity is also discussed in this paper.      

Nonlinear separation of show-through image mixtures using a physical model trained with ICA

Often, when we scan a document, the image from the back page shows through, due to partial transparency of the paper, giving rise to a mixture of two images. We address the problem of separating these images through the use of a physical model of the mixture process. The model is nonlinear but invertible, and we use the inverse model to perform the separation. The model is trained through the MISEP technique of nonlinear ICA. Bounded independent sources are proved to be separable through this method, apart from offset, scale and permutation indeterminacies.We compare our results with those obtained with other approaches and with different separation models that were trained with MISEP. For the latter case we test a bilinear model and MLP-based models, using both symmetry-based regularization and the more recently proposed minimal nonlinear distortion regularization. Quantitative quality measures show that the approach that we propose is superior to the other methodologies.     

Blind separation of instantaneous mixtures of nonstationary sources

Most source separation algorithms are based on a model of stationary sources. However, it is a simple matter to take advantage of possible nonstationarities of the sources to achieve separation. This paper develops novel approaches in this direction based on the principles of maximum likelihood and minimum mutual information. These principles are exploited by efficient algorithms in both the off-line case (via a new joint diagonalization procedure) and in the on-line case (via a Newton-like procedure). Some experiments showing the good performance of our algorithms and evidencing an interesting feature of our methods are presented: their ability to achieve a kind of super-efficiency. The paper concludes with a discussion contrasting separating methods for non-Gaussian and nonstationary models and emphasizing that, as a matter of fact, “what makes the algorithms work” is-strictly speaking-not the nonstationarity itself but rather the property that each realization of the source signals has a time-varying envelope     

Convolutive blind source separation based on joint block Toeplitzation and block-inner diagonalization

This paper takes a close look at the block Toeplitz structure and block-inner diagonal structure of auto correlation matrices of source signals in convolutive blind source separation (BSS) problems. The aim is to propose a one-stage time-domain algorithm for convolutive BSS by explicitly exploiting the structure in autocorrelation matrices of source signals at different time delays and inherent relations among these matrices. The main idea behind the proposed algorithm is to implement the joint block Toeplitzation and block-inner diagonalization (JBTBID) of a set of correlation matrices of the observed vector sequence such that the mixture matrix can be extracted. For this purpose, a novel tri-quadratic cost function is introduced. The important feature of this tri-quadratic contrast function enables the development of an efficient algebraic method based on triple iterations for searching the minimum point of the cost function, which is called the triply iterative algorithm (TIA). Through the cyclic minimization process in the proposed TIA, it is expected that the JBTBID is achieved. The source signals can be retrieved. Moreover, the asymptotic convergence of the proposed TIA is analyzed. Convergence performance of the TIA and the separation results are also demonstrated by simulations in comparison with some other prominent two-stage time-domain methods.     

Time-domain period detection in short-duration videos

This paper is concerned with detecting the period of cyclic object motion in a short video or sequence with a limited number of frames. This problem can be studied with either frequency-domain methods or time-domain methods. A frequency-domain method is fundamentally limited in terms of frequency resolution—especially with a small number of frames—and its ability to handle a periodic impulsive or spiky signal. Existing time-domain methods are primarily based on an analysis of the autocorrelation function of a signal and can be sensitive to noise in the signal. In this paper, we offer an alternative time-domain method. Rather than using autocorrelation as the basis, our proposed method uses peak analysis. Specifically, after computing the similarity between a reference image and those in the sequence, our algorithm applies one of two period detection procedures—one based on clustering and the other on watershed to analyze the peaks of the similarity time series—in estimating the period of object motion embedded in the similarity function. Video sequences from three different applications are used to establish the feasibility of our proposed algorithm and its superiority to competing algorithms.     

Adaptive nonlinear PCA algorithms for blind source separation without prewhitening

Blind source separation (BSS) aims at recovering statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Besides independent component analysis, nonlinear principal component analysis (NPCA) is shown to be another useful tool for solving this problem, but it requires that the measured data be prewhitened. By taking into account the autocorrelation matrix of the measured data, we present in this paper a modified NPCA criterion, and develop a least-mean-square (LMS) algorithm and a recursive least-squares algorithm. They can perform the online BSS using directly the unwhitened observations. Since a natural gradient learning is applied and the prewhitening process is removed, the proposed algorithms work more efficiently than the existing NPCA algorithms, as verified by computer simulations on man-made sources as well as practical speech signals.