Adaptive weighted orthogonal constrained algorithm for blind source separation

Blind source separation (BSS) consists of recovering the statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Pre-whitening is a useful pre-processing technique in BSS. However, BSS algorithms based on the pre-whitened data lack the equivariance property, one of the significant properties in BSS. By transforming the pre-whitening into a weighted orthogonal constraint condition, this paper proposes a new definition of the contrast function. In light of the constrained optimization method, various weighted orthogonal constrained BSS algorithms with equivariance property are developed. Simulations on man-made signals and practical speech signals show the proposed weighted orthogonal constrained BSS algorithms have better separation ability, convergent speed and steady state performance.     

M-estimator with asymmetric influence function for estimating the Burr type III parameters with outliers

The Burr type III distribution allows for a wider region for the skewness and kurtosis plane, which covers several distributions including the log-logistic, and the Weibull and Burr type XII distributions. However, outliers may occur in the data set. The robust regression method such as an M-estimator with symmetric influence function has been successfully used to diminish the effect of outliers on statistical inference. However, when the data distribution is asymmetric, these methods yield biased estimators. We present an M-estimator with asymmetric influence function (AM-estimator) based on the quantile function of the Burr type III distribution to estimate the parameters for complete data with outliers. The simulation results show that the M-estimator with asymmetric influence function generally outperforms the maximum likelihood and traditional M-estimator methods in terms of the bias and root mean square errors. One real example is used to demonstrate the performance of our proposed method.     

Markov and Semi-Markov switching of source appearances for nonstationary independent component analysis.

Independent component analysis (ICA) is currently the most popularly used approach to blind source separation (BSS), the problem of recovering unknown source signals when their mixtures are observed but the actual mixing process is unknown. Many ICA algorithms assume that a fixed set of source signals consistently exists in mixtures throughout the time-series to be examined. However, real-world signals often have such difficult nonstationarity that each source signal abruptly appears or disappears, thus the set of active sources dynamically changes with time. In this paper, we propose switching ICA (SwICA), which focuses on such situations. The proposed approach is based on the noisy ICA formulated as a generative model. We employ a special type of hidden Markov model (HMM) to represent such prior knowledge that the source may abruptly appear or disappear with time. The special HMM setting t hen provides an effect ofvariable selection in a dynamic way. We use the variational Bayes (VB) method to derive an effective approximation of Bayesian inference for this model. In simulation experiments using artificial and realistic source signals, the proposed method exhibited performance superior to existing methods, especially in the presence of noise. The compared methods include the natural-gradient ICA with a nonholonomic constraint, and the existing ICA method incorporating an HMM source model, which aims to deal with general nonstationarities that may exist in source signals. In addition, the proposed method could successfully recover the source signals even when the total number of true sources was overestimated or was larger than that of mixtures. We also propose a modification of the basic Markov model into a semi-Markov model, and show that the semi-Markov one is more effective for robust estimation of the source appearance.     

A Class of Frobenius Norm-Based Algorithms Using Penalty Term and Natural Gradient for Blind Signal Separation

We consider the blind signal separation (BSS) problem of instantaneous mixtures using penalty term and natural gradient. A class of Frobenius norm-based algorithms consisting of the offline/block processing (BP), online processing (OP) algorithms, and their normalized versions is proposed for separating nonstationary and nonwhite signals. The BP and OP algorithms, respectively, suitable for blind separation with offline and online data, are derived by using the nonstationarity and nonwhiteness of signals and the natural gradient method in conjunction with an appropriate penalty term. Associated with almost all algorithms employing a gradient method is a gradient noise problem. We thus develop, from BP and OP, their normalized versions in which the update of an unknown demixing matrix is based on the minimal disturbance principle. We show that the resulting updates are in the same direction as those of the original algorithms but with a scaling factor whose upper bound is unity. Algorithms using the nonstationarity and nonwhiteness properties have been proposed before but, due to the use of logarithms in their derivation, they are not capable of separating signals that are not persistently active and require regularization parameters to mitigate the problem. In this paper, the superior performance of the proposed algorithms to the previously proposed logarithm-based algorithms with and without regularization when separating nonpersistently active source signals is presented through some illustrative numerical experiments.     

Genetic algorithms based robust frequency estimation of sinusoidal signals with stationary errors

In this paper, we consider the fundamental problem of frequency estimation of multiple sinusoidal signals with stationary errors. We propose genetic algorithm and outlier-insensitive criterion function based technique for the frequency estimation problem. In the simulation studies and real life data analysis, it is observed that the proposed genetic algorithm based robust frequency estimators are able to resolve frequencies of the sinusoidal model with high degree of accuracy. Among the proposed methods, the genetic algorithm based least squares estimator, in the no-outlier scenario, provides efficient estimates, in the sense that their mean square errors attain the corresponding Cramér-Rao lower bounds. In the presence of outliers, the proposed robust methods perform quite well and seem to have a fairly high breakdown point with respect to level of outlier contamination. The proposed methods significantly do not depend on the initial guess values required for other iterative frequency estimation methods.     

Blind Source Separation of Postnonlinear Convolutive Mixture

In this paper, a novel solution is developed to solve blind source separation of postnonlinear convolutive mixtures. The proposed model extends the conventional linear instantaneous mixture model to include both convolutive mixing and postnonlinear distortion. The maximum-likelihood (ML) approach solution based on the expectation-maximization (EM) algorithm is developed to estimate the source signals and the parameters in the proposed nonlinear model. In the proposed solution, the sufficient statistics associated with the source signals are estimated in the E-step, while the model parameters are optimized through these statistics in the M-step. However, the complication resulted from the postnonlinear function associated with the mixture renders these statistics difficult to be formulated in a closed form and hence causes intractability in the parameter optimization. A computationally efficient algorithm is proposed which uses the extended Kalman smoother (EKS) to facilitate the E-step tractable and a set of self-updated polynomials is used as the nonlinearity estimator to facilitate closed form estimations of the parameters in the M-step. The theoretical foundation of the proposed solution has been rigorously developed and discussed in details. Both simulations and recorded speech signals have been carried out to verify the success and efficacy of the proposed algorithm. Remarkable improvement has been obtained when compared with the existing algorithms.     

Adaptive algorithm for blind separation from noisy time-varying mixtures

This article addresses the problem of blind source separation from time-varying noisy mixtures using a state variable model and recursive estimation. An estimate of each source signal is produced real time at the arrival of new observed mixture vector. The goal is to perform the separation and attenuate noise simultaneously, as well as to adapt to changes that occur in the mixing system. The observed data are projected along the eigenvectors in signal subspace. The subspace is tracked real time. Source signals are modeled using low-order AR (autoregressive) models, and noise is attenuated by trading off between the model and the information provided by measurements. The type of zero-memory nonlinearity needed in separation is determined on-line. Predictor-corrector filter structures are proposed, and their performance is investigated in simulation using biomedical and communications signals at different noise levels and a time-varying mixing system. In quantitative comparison to other widely used methods, significant improvement in output signal-to-noise ratio is achieved.     

Maximum trimmed likelihood estimator for multivariate mixed continuous and categorical data

In this article, we apply the maximum trimmed likelihood (MTL) approach [Hadi, A.S., Luceño, A., 1997. Maximum trimmed likelihood estimators: a unified approach, examples, and algorithms. Comput. Statist. Data Anal. 25, 251-272] to obtain the robust estimators of multivariate location and shape, especially for data mixed with continuous and categorical variables. The forward search algorithm [Atkinson, A.C., 1994. Fast very robust methods for the detection of multiple outliers. J. Amer. Statist. Assoc. 89, 1329-1339] is adapted to compute the proposed MTL estimates. A simulation study shows that the proposed estimator outperforms the classical maximum likelihood estimator when outliers exist in data. Real data sets are also used to illustrate the method and results of the detection of the outliers.     

Entropy based robust estimator and its application to line-based mapping

This paper presents a robust mapping algorithm for an application in autonomous robots. The method is inspired by the notion of entropy from information theory. A kernel density estimator is adopted to estimate the appearance probability of samples directly from the data. An Entropy Based Robust (EBR) estimator is then designed that selects the most reliable inliers of the line segments. The inliers maintained by the entropy filter are those samples that carry more information. Hence, the parameters extracted from EBR estimator are accurate and robust to the outliers. The performance of the EBR estimator is illustrated by comparing the results with the performance of three other estimators via simulated and real data.     

Robust estimation of dimension reduction space

Most dimension reduction methods based on nonparametric smoothing are highly sensitive to outliers and to data coming from heavy-tailed distributions. Two recently proposed methods, minimum average variance estimation and outer product of gradients, can be and are made robust in such a way that preserves all advantages of the original approach. Their extension based on the local one-step M-estimators is sufficiently robust to outliers and data from heavy-tailed distributions, it is relatively easy to implement, and surprisingly, it performs as well as the original methods when applied to normally distributed data.     

Robust learning algorithm for multiplicative neuron model artificial neural networks

The two most commonly used types of artificial neural networks (ANNs) are the multilayer feed-forward and multiplicative neuron model ANNs. In the literature, although there is a robust learning algorithm for the former, there is no such algorithm for the latter. Because of its multiplicative structure, the performance of multiplicative neuron model ANNs is affected negatively when the dataset has outliers. On this issue, a robust learning algorithm for the multiplicative neuron model ANNs is proposed that uses Huber's loss function as fitness function. The training of the multiplicative neuron model is performed using particle swarm optimization. One principle advantage of this algorithm is that the parameter of the scale estimator, which is an important factor affecting the value of Huber's loss function, is also estimated with the proposed algorithm. To evaluate the performance of the proposed method, it is applied to two well-known real world time series datasets, and also a simulation study is performed. The algorithm has superior performance both when it is applied to real world time series datasets and the simulation study when compared with other ANNs reported in the literature. Another of its advantages is that, for datasets with outliers, the results are very close to the results obtained from the original datasets. In other words, we demonstrate that the algorithm is unaffected by outliers and has a robust structure.     

基于新型混合模型的欠定盲分离方法

针对欠定盲分离问题,提出了一种新的源恢复方法. 在时频域局部区域采用复高斯分布对源信号进行建模,将语音信号的稀疏性和局部平稳性结合在一起,提出了一种新的混合模型来描述观测信号在局部区域的概率分布.通过该模型,将每个时频点的源信号状态的判断问题转换成模型的参数估计和后验概率的计算问题,最后通过子混合矩阵的逆恢复出源信号. 实验结果表明,该方法具有很快的收敛速度,并且比已有方法具有更好的分离性能.     

Performance Estimation of Speech Recognition System Under Noise Conditions Using Objective Quality Measures and Artificial Voice

It is essential to ensure quality of service (QoS) when offering a speech recognition service for use in noisy environments. This means that the recognition performance in the target noise environment must be investigated. One approach is to estimate the recognition performance from a distortion value, which represents the difference between noisy speech and its original clean version. Previously, estimation methods using the segmental signal-to-noise ratio (SNRseg), the cepstral distance (CD), and the perceptual evaluation of speech quality (PESQ) have been proposed. However, their estimation accuracy has not been verified for the case when a noise reduction algorithm is adopted as a preprocessing stage in speech recognition. We, therefore, evaluated the effectiveness of these distortion measures by experiments using the AURORA-2J connected digit recognition task and four different noise reduction algorithms. The results showed that in each case the distortion measure correlates well with the word accuracy when the estimators used are optimized for each individual noise reduction algorithm. In addition, it was confirmed that when a single estimator, optimized for all the noise reduction algorithms, is used, the PESQ method gives a more accurate estimate than SNRseg and CD. Furthermore, we have proposed the use of artificial voice of several seconds duration instead of a large amount of real speech and confirmed that a relatively accurate estimate can be obtained by using the artificial voice.     

Source separation using regularized NMF with MMSE estimates under GMM priors with online learning for the uncertainties

We propose a new method to incorporate priors on the solution of nonnegative matrix factorization (NMF). The NMF solution is guided to follow the minimum mean square error (MMSE) estimates of the weight combinations under a Gaussian mixture model (GMM) prior. The proposed algorithm can be used for denoising or single-channel source separation (SCSS) applications. NMF is used in SCSS in two main stages, the training stage and the separation stage. In the training stage, NMF is used to decompose the training data spectrogram for each source into a multiplication of a trained basis and gains matrices. In the separation stage, the mixed signal spectrogram is decomposed as a weighted linear combination of the trained basis matrices for the source signals. In this work, to improve the separation performance of NMF, the trained gains matrices are used to guide the solution of the NMF weights during the separation stage. The trained gains matrix is used to train a prior GMM that captures the statistics of the valid weight combinations that the columns of the basis matrix can receive for a given source signal. In the separation stage, the prior GMMs are used to guide the NMF solution of the gains/weights matrices using MMSE estimation. The NMF decomposition weights matrix is treated as a distorted image by a distortion operator, which is learned directly from the observed signals. The MMSE estimate of the weights matrix under the trained GMM prior and log-normal distribution for the distortion is then found to improve the NMF decomposition results. The MMSE estimate is embedded within the optimization objective to form a novel regularized NMF cost function. The corresponding update rules for the new objectives are derived in this paper. The proposed MMSE estimates based regularization avoids the problem of computing the hyper-parameters and the regularization parameters. MMSE also provides a better estimate for the valid gains matrix. Experimental results show that the proposed regularized NMF algorithm improves the source separation performance compared with using NMF without a prior or with other prior models.     

Robust fitting of mixtures using the trimmed likelihood estimator

The maximum likelihood estimator (MLE) has commonly been used to estimate the unknown parameters in a finite mixture of distributions. However, the MLE can be very sensitive to outliers in the data. In order to overcome this the trimmed likelihood estimator (TLE) is proposed to estimate mixtures in a robust way. The superiority of this approach in comparison with the MLE is illustrated by examples and simulation studies. Moreover, as a prominent measure of robustness, the breakdown point (BDP) of the TLE for the mixture component parameters is characterized. The relationship of the TLE with various other approaches that have incorporated robustness in fitting mixtures and clustering are also discussed in this context.     

Feature‐Preserving Surface Reconstruction From Unoriented,Noisy Point Data

We propose a robust method for surface mesh reconstruction from unorganized, unoriented, noisy and outlier‐ridden 3D point data. A kernel‐based scale estimator is introduced to estimate the scale of inliers of the input data. The best tangent planes are computed for all points based on mean shift clustering and adaptive scale sample consensus, followed by detecting and removing outliers. Subsequently, we estimate the normals for the remaining points and smooth the noise using a surface fitting and projection strategy. As a result, the outliers and noise are removed and filtered, while the original sharp features are well preserved. We then adopt an existing method to reconstruct surface meshes from the processed point data. To preserve sharp features of the generated meshes that are often blurred during reconstruction, we describe a two‐step approach to effectively recover original sharp features. A number of examples are presented to demonstrate the effectiveness and robustness of our method.     

基于特征值分解的循环平稳离散时间信号的盲分离

盲分离的目的是从观测到的混叠信号中恢复出各个未知的源信号,现今的很多方法都是利用了信号时域表示的某些统计特性来解决这个问题。从信号频域分析的角度提出了一种利用信号的循环平稳特性来处理离散时间信号的频域盲分离方法。该方法构造两个二阶统计矩阵的乘积,并对该乘积矩阵进行特征值分解,从而实现源信号的分离;同时,还对特征值分解的条件进行了分析。该方法在低维信号的情况下可以取得相当满意的分离效果,仿真结果表明该方法具有良好的性能。     

基于子带分解的语音分离算法研究

提出一种有效解决不相互独立语音源信号混合的分离算法.利用子带分解方法,将混合信号分解成多个子带信号,在各个子带上分别进行语音分离得出语音分离信号,利用提出的相关性能指数,判断出相互独立的子带信号,把该子带的分离矩阵作为混合信号的解混合矩阵对混合信号进行分离.实验证明了本算法对相关语音源信号较好的分离效果.     

Impact of Distance Measures on the Performance of AIS Data Clustering

Automatic Identification System (AIS) data stream analysis is based on the AIS data of different vessel’s behaviours, including the vessels’ routes. When the AIS data consists of outliers, noises, or are incomplete, then the analysis of the vessel’s behaviours is not possible or is limited. When the data consists of outliers, it is not possible to automatically assign the AIS data to a particular vessel. In this paper, a clustering method is proposed to support the AIS data analysis, to qualify noises and outliers with respect to their suitability, and finally to aid the reconstruction of the vessel’s trajectory. In this paper, clustering results have been obtained using selected algorithms, including k-means, k-medoids, and fuzzy c-means. Based on the clustering results, it is possible to decide on the qualification of data with outliers and on their usefulness in the reconstruction of the vessel trajectory. The main aim of this paper is to answer how different distance measures during a clustering process can influence AIS data clustering quality. The main core question is whether or not they have an impact on the process of reconstruction of the vessel trajectories when the data are damaged. The research question during the computational experiments asked whether or not distance measure influence AIS data clustering quality. The computational experiments have been carried out using original AIS data. In general, the experiment and the results confirm the usefulness of the cluster-based analysis when the data include outliers that are derived from the natural environment. It is also possible to monitor and to analyse AIS data using clustering when the data include outliers. The computational experiment results confirm that the k-means with Euclidean distance has the best performance.     

Estimation of fractional integration in the presence of data noise

A comparative study is presented regarding the performance of commonly used estimators of the fractional order of integration when data is contaminated by noise. In particular, measurement errors, additive outliers, temporary change outliers, and structural change outliers are addressed. It occurs that when the sample size is not too large, as is frequently the case for macroeconomic data, then non-persistent noise will generally bias the estimators of the memory parameter downwards. On the other hand, relatively more persistent noise like temporary change outliers and structural changes can have the opposite effect and thus bias the fractional parameter upwards. Surprisingly, with respect to the relative performance of the various estimators, the parametric conditional maximum likelihood estimator with modelling of the short run dynamics clearly outperforms the semiparametric estimators in the presence of noise that is not too persistent. However, when a non-zero mean is allowed for, it may reverse the conclusion.