A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals

Capturing statistical regularities in complex, high-dimensional data is an important problem in machine learning and signal processing. Models such as principal component analysis (PCA) and independent component analysis (ICA) make few assumptions about the structure in the data and have good scaling properties, but they are limited to representing linear statistical regularities and assume that the distribution of the data is stationary. For many natural, complex signals, the latent variables often exhibit residual dependencies as well as nonstationary statistics. Here we present a hierarchical Bayesian model that is able to capture higher-order nonlinear structure and represent nonstationary data distributions. The model is a generalization of ICA in which the basis function coefficients are no longer assumed to be independent; instead, the dependencies in their magnitudes are captured by a set of density components. Each density component describes a common pattern of deviation from the marginal density of the pattern ensemble; in different combinations, they can describe nonstationary distributions. Adapting the model to image or audio data yields a nonlinear, distributed code for higher-order statistical regularities that reflect more abstract, invariant properties of the signal.     

基于FastICA的语音分离与图像分离

独立分量分析0CA）基于信号的高阶统计量,能从混合信号中分离出既具有统计独立性又具有非高斯性的源信号,在诸多ICA算法中,固定点算法（也称FastICA）以其收敛速度快、分离效果好被广泛应用于信号处理领域。在介绍ICA的基本模型与FastICA算法的原理后,分别对混合的语音信号与图像信号进行了分离实验,仿真结果表明FastICA应用于语音分离与图像分离,效果都很好。     

Minimax mutual information approach for independent component analysis

Minimum output mutual information is regarded as a natural criterion for independent component analysis (ICA) and is used as the performance measure in many ICA algorithms. Two common approaches in information-theoretic ICA algorithms are minimum mutual information and maximum output entropy approaches. In the former approach, we substitute some form of probability density function (pdf) estimate into the mutual information expression, and in the latter we incorporate the source pdf assumption in the algorithm through the use of nonlinearities matched to the corresponding cumulative density functions (cdf). Alternative solutions to ICA use higher-order cumulant-based optimization criteria, which are related to either one of these approaches through truncated series approximations for densities. In this article, we propose a new ICA algorithm motivated by the maximum entropy principle (for estimating signal distributions). The optimality criterion is the minimum output mutual information, where the estimated pdfs are from the exponential family and are approximate solutions to a constrained entropy maximization problem. This approach yields an upper bound for the actual mutual information of the output signals - hence, the name minimax mutual information ICA algorithm. In addition, we demonstrate that for a specific selection of the constraint functions in the maximum entropy density estimation procedure, the algorithm relates strongly to ICA methods using higher-order cumulants.     

Independent component analysis based on nonparametric density estimation

In this paper, we introduce a novel independent component analysis (ICA) algorithm, which is truly blind to the particular underlying distribution of the mixed signals. Using a nonparametric kernel density estimation technique, the algorithm performs simultaneously the estimation of the unknown probability density functions of the source signals and the estimation of the unmixing matrix. Following the proposed approach, the blind signal separation framework can be posed as a nonlinear optimization problem, where a closed form expression of the cost function is available, and only the elements of the unmixing matrix appear as unknowns. We conducted a series of Monte Carlo simulations, involving linear mixtures of various source signals with different statistical characteristics and sample sizes. The new algorithm not only consistently outperformed all state-of-the-art ICA methods, but also demonstrated the following properties: 1) Only a flexible model, capable of learning the source statistics, can consistently achieve an accurate separation of all the mixed signals. 2) Adopting a suitably designed optimization framework, it is possible to derive a flexible ICA algorithm that matches the stability and convergence properties of conventional algorithms. 3) A nonparametric approach does not necessarily require large sample sizes in order to outperform methods with fixed or partially adaptive contrast functions.     

基于方差滤波和ICA算法的人眼定位

采用方差滤波器确定人眼候选区域, 结合独立成分分析(ICA)方法对人眼进行快速定位, 能够有效地提取训练图像的高阶统计特征, 很好地去除基向量的相关性. 与主成分分析(PCA)和传统ICA方法相比具有更好的鉴别能力. 实验表明, 该方法的识别率可达97.3%, 并对光照和姿态变化也具有很好的鲁棒性.     

Nonlinear and extra-classical receptive field properties and the statistics of natural scenes

The neural mechanisms of early vision can be explained in terms of an information-theoretic optimization of the neural processing with respect to the statistical properties of the natural environment. Recent applications of this approach have been successful in the prediction of the linear filtering properties of ganglion cells and simple cells, but the relations between the environmental statistics and cortical nonlinearities, like those of end-stopped or complex cells, are not yet fully understood. Here we present extensions of our previous investigations of the exploitation of higher-order statistics by nonlinear neurons. We use multivariate wavelet statistics to demonstrate that a strictly linear processing would inevitably leave substantial statistical dependencies between the outputs of the units. We then consider how the basic nonlinearities of cortical neurons--gain control and ON/OFF half-wave rectification--can exploit these higher-order statistical dependencies. We first show that gain control provides an adaptation to the polar separability of the multivariate probability density function (PDF), and, together with an output nonlinearity, enables an overcomplete sparse coding. We then consider how the remaining higher-order dependencies between different units can be exploited by a combination of basic ON/OFF point nonlinearities and subsequent weighted linear combinations. We consider two statistical optimization schemes for the computation of the optimal weights: principal component analysis (PCA) and independent component analysis (ICA). Since the intermediate nonlinearities transform some of the higher-order dependencies into second-order dependencies even the basic PCA approach is able to exploit part of the redundancies. ICA ignores this second-order structure, but can exploit higher-order dependencies. Both schemes yield a variety of nonlinear units which comprise the typical nonlinear processing properties, such as end-stopping, side-stopping, complex-cell properties and extra-classical receptive field properties, but the 'ideal' complex cells seem only to occur with PCA. Thus, a combination of ON/OFF nonlinearities with an integrated PCA-ICA strategy seems necessary to exploit the statistical properties of natural images.     

Linear Multilayer ICA Using Adaptive PCA

Linear multilayer independent component analysis (LMICA) is an approximate algorithm for ICA. In LMICA, approximate independent components are efficiently estimated by optimizing only highly dependent pairs of signals when all the sources are super-Gaussian. In this paper, the nonlinear functions in LMICA are generalized, and a new method using adaptive PCA is proposed for the selection of pairs of highly dependent signals. In this method, at first, all the signals are sorted along the first principal axis of their higher-order correlation matrix. Then, the sorted signals are divided into two groups so that relatively highly correlated signals are collected in each group. Lastly, each of them is sorted recursively. This process is repeated until each group consists of only one or two signals. Because a well-known adaptive PCA algorithm named PAST is utilized for calculating the first principal axis, this method is quite simple and efficient. Some numerical experiments verify the effectiveness of LMICA with this improvement.     

基于CEEMDAN-ICA的单通道脑电信号眼电伪迹滤除方法

传统盲源分离法不能解决欠定问题,且分离信号与源信号对应关系不确定.提出一种基于自适应噪声完备经验模态分解(CEEMDAN)和独立成分分析(ICA)相结合的脑电信号眼电伪迹自动去除方法.该方法首先将含伪迹脑电信号自适应分解成多维本征模态函数(IMF),以满足盲源分离方法对信号正定或超定要求,再对本征模态函数用ICA方法构建多维源信号,最后利用模糊熵阈值判据判别多维源信号中的伪迹信号,完成滤波并重构脑电信号.该方法相比于其他算法,能更好的去除眼电伪迹并保留原始信息,适合单通道脑电信号预处理.     

独立分量分析在图像去噪方面的应用

独立分量分析(independent component analysis,ICA)是基于信号高阶统计量的盲源分离方法。在分析独立分量分析的基本模型及方法的基础上,讨论了有噪信号的独立分量分析(Noisy ICA),结合传统有噪图像分离方法与结合改进FastlCA算法有噪图像分离仿真研究进行对比。结果表明,该算法即使在高水平噪声图像中,也能够分理出比较清晰的图像。     

基于ICA的白噪声消除方法研究

为了消除润滑油内金属磨粒检测系统(metal debris detection system,MDDS)输出信号中混合的高斯白噪声,提出了一个基于ICA的算法对两路输出信号进行消噪处理.对两路信号添加前缀信号,并按照所述步骤进行两次ICA后得到三路独立源信号,根据ICA前后前缀信号幅值和相位的变化校正ICA分离结果的幅值和相位,完全恢复源信号.对MDDS的输出信号进行仿真以验证算法的去噪效果,实验结果表明,该算法可以有效地消除输出信号中的白噪声.     

基于curvelet变换和独立分量分析的含噪盲源分离

独立分量分析（ICA）是基于信号高阶统计量的盲源分离方法,在高阶统计量方法中,由于高斯信号的高阶累计量为零,所以系统存在加性高斯噪声时就难以处理。提出了一种基于curvelet阈值去噪和FastICA算法的含噪信号盲分离的方法,并对高斯噪声环境下的混合图像进行了盲分离的仿真。结果表明,该方法能很好地解决由于存在加性高斯噪声而导致经典ICA算法性能发生严重恶化的问题;同时将curvelet变换去噪应用于含噪图像的盲源分离中,可以提高混合图像的信噪比,相对于小波去噪后的ICA算法,其分离性能有很大改善。     

A new method for fMRI data processing: Neighborhood independent component correlation algorithm and its preliminary application

Independent component analysis (ICA) is a newly developed promising technique in signal processing applications. The effective separation and discrimination of functional Magnetic Resonance Imaging (fMRI) signals is an area of active research and widespread interest. Therefore, the development of an ICA based fMRI data processing method is of obvious value both theoretically and in potential applications. In this paper, analyzed firstly is the drawback of the extant popular ICA-fMRI method where the adopted signal model assumes the independence of spatial distributions of the signals and noise. Then presented is a new fMRI signal model, which assumes the independence of temporal courses of signal and noise in a tiny spatial domain. Consequently we get a novel fMRI data processing method: Neighborhood independent component correlation algorithm. The effectiveness is elucidated through theoretical analysis and simulation tests, and finally a real fMRI data test is presented.     

Dual multivariate auto-regressive modeling in state space for temporal signal separation

Many existing independent component analysis (ICA) approaches result in deteriorated performance in temporal source separation because they have not taken into consideration of the underlying temporal structure of sources. In this paper, we model temporal sources as a general multivariate auto-regressive (AR) process whereby an underlying multivariate AR process in observation space is obtained. In this dual AR modeling, the mixing process from temporal sources to observations is the same as the mixture from the nontemporal residuals of the source AR (SAR) process to that of the observation AR (OAR) process. We can therefore avoid the source temporal effects in performing ICA by learning the demixing system on the independently distributed OAR residuals rather than the time-correlated observations. Particularly, we implement this approach by modeling each source signal as a finite mixture of generalized autoregressive conditional heteroskedastic (GARCH) process. The adaptive algorithms are proposed to extract the OAR residuals appropriately online, together with learning the demixing system via a nontemporal ICA algorithm. The experiments have shown its superior performance on temporal source separation.     

基于最短路径和自然梯度的过完备ICA算法

独立成分分析(ICA)是一种在给出的随机向量中找出统计独立的数据的统计方法，而过完备独立成分分析则是ICA问题中的一类特殊的情形，它要的源信号的数目比观测信号的数目要多。该文提出了一种基于最短路径算法和自然梯度的解决过完备独立成分分析的新算法Turbo-overcomplete。该算法采用了最短路径方法来推断源信号和采用自然梯度的方法来学习基向量，并采用Turbo-overcomplete算法来进行语音信号分离的实验，并把实验结果与现在的一些过完备独立成份分析算法进行了比较。     

Ensemble learning for independent component analysis

It is well known that the applicability of independent component analysis (ICA) to high-dimensional pattern recognition tasks such as face recognition often suffers from two problems. One is the small sample size problem. The other is the choice of basis functions (or independent components). Both problems make ICA classifier unstable and biased. In this paper, we propose an enhanced ICA algorithm by ensemble learning approach, named as random independent subspace (RIS), to deal with the two problems. Firstly, we use the random resampling technique to generate some low dimensional feature subspaces, and one classifier is constructed in each feature subspace. Then these classifiers are combined into an ensemble classifier using a final decision rule. Extensive experimentations performed on the FERET database suggest that the proposed method can improve the performance of ICA classifier.     

Performance study of receiver based on independent component analysis in CDMA systems

To improve the performances of the wireless mobile communication system, a statistical method is illustrated in this paper. It consists in separating the signals at the reception of communication systems based on code division multiple access technology. The idea is to optimize the separation of the various users sharing the same frequency and temporal resources using the emergent statistical method of independent component analysis (ICA). ICA makes it possible to extract emitted signals that are as statistically independent as possible. The bit error rate and the signal to noise ratio are used as criteria for evaluating the performances of the ICA receiver. Adding white Gaussian noise to the input signal channel and the Rayleigh channel (fading channel) cases has been considered. A comparative study with conventional receivers such as the RAKE, the matched filter (MF), and the minimum mean-squared error is carried out. The obtained results show the superiority of an ICA receiver compared to an MF receiver.     

基于高阶累计统计量的微生物发酵过程监测

传统多向核独立成分分析(MKICA)方法的实质是把基于独立成分分析(ICA)中的白化处理主元分析(PCA)替换为核主元分析(KPCA)后利用二阶统计量进行过程监控,并未利用过程数据的阶段特性和高阶累积量信息,为了解决此问题,提出高阶累积量分析(HCA)与多向核熵独立成份分析(MKECA)相结合的多向高阶累计量的核熵独立成分分析方法(HCA-MKEICA).首先,采用核熵独立成份分析(KECA)对原始数据进行数据转换,解决数据的非线性;然后,在高维核熵空间利用HCA技术构建新的统计量用于过程监控;最后,将该方法应用于青霉素仿真平台和实际的工业过程并与MKICA方法进行对比,以验证所提出方法的有效性.     

有限制的MFA-ICA的算法及其在图像特征提取中的应用

统的独立成分分析(IndependentComponentAnalysis,ICA)是一种无噪声模型,而实际应用中噪声是存在的。根据多元统计中的因子分析模型,改变其假设条件,从而得到一种有噪声ICA模型,对于模型参数,引入平均场近似(MeanFieldApproximation,MFA)原理来求解。针对图像特征提取,通过增加对模型参数的一些限制,使其能得到更为独立的图像特征,为图像识别提供更可靠的特征信息,从而大大提高识别率。通过仿真模拟图形以及ORL人脸数据进行实验,将传统的独立成分分析算法、无限制的MFA ICA算法以及增加限制条件的MFA ICA算法进行比较,从仿真模拟图形实验结果看,限制的MFA ICA算法能分离出更独立的特征,同时利用限制的MFA ICA算法识别效果明显优于传统ICA算法和无限制MFA ICA算法。     

A new independent component analysis for speech recognition and separation

This paper presents a novel nonparametric likelihood ratio (NLR) objective function for independent component analysis (ICA). This function is derived through the statistical hypothesis test of independence of random observations. A likelihood ratio function is developed to measure the confidence toward independence. We accordingly estimate the demixing matrix by maximizing the likelihood ratio function and apply it to transform data into independent component space. Conventionally, the test of independence was established assuming data distributions being Gaussian, which is improper to realize ICA. To avoid assuming Gaussianity in hypothesis testing, we propose a nonparametric approach where the distributions of random variables are calculated using kernel density functions. A new ICA is then fulfilled through the NLR objective function. Interestingly, we apply the proposed NLR-ICA algorithm for unsupervised learning of unknown pronunciation variations. The clusters of speech hidden Markov models are estimated to characterize multiple pronunciations of subword units for robust speech recognition. Also, the NLR-ICA is applied to separate the linear mixture of speech and audio signals. In the experiments, NLR-ICA achieves better speech recognition performance compared to parametric and nonparametric minimum mutual information ICA.     

Independent component analysis: A flexible nonlinearity and decorrelating manifold approach

Independent component analysis (ICA) finds a linear transformation to variables that are maximally statistically independent. We examine ICA and algorithms for finding the best transformation from the point of view of maximizing the likelihood of the data. In particular, we discuss the way in which scaling of the unmixing matrix permits a "static" nonlinearity to adapt to various marginal densities. We demonstrate a new algorithm that uses generalized exponential functions to model the marginal densities and is able to separate densities with light tails. We characterize the manifold of decorrelating matrices and show that it lies along the ridges of high-likelihood unmixing matrices in the space of all unmixing matrices. We show how to find the optimum ICA matrix on the manifold of decorrelating matrices, and as an example we use the algorithm to find independent component basis vectors for an ensemble of portraits.