Weighted part mutual information related component analysis for quality-related process monitoring

Industrial products have become the core of today’s highly competitive international society, but quality-related faults happened in practical industrial processes heavily affect product quality. In this paper, we will consider the problem of the detection of quality-related faults. Inspired by part mutual information (PMI), we develop a process monitoring method called weighted PMI based related component analysis (WPMI-RCA). Firstly, combining PMI and Bayesian weighted fusion, process variables strongly related to quality are selected with the supervision of multi-quality indicators. Then, the selected variables are modeled by related component analysis (RCA) and thus orthogonal related components (RCs) containing the main information of quality variations can be obtained. The process data space can be divided into two subspaces and the monitoring statistics are developed for the quality-related fault detection. Finally, the validity of WPMI-RCA is demonstrated by a numerical example and the benchmark Tennessee Eastman process (TEP). The proposed method can improve the detection rates of quality-related faults and significantly reduce the nuisance detections. It may be helpful to improve the management efficiency for practical industrial processes.     

A simplified approach to independent component analysis

Independent Component Analysis (ICA) is one of the fastest growing fields in the area of neural networks and signal processing. Blind Source Separation (BSS) is one of the applications of ICA. In this paper, ICA has been used for separating unknown source signals. BSS is used to extract independent signal components from their observed linear mixtures at an array of sensors. Various statistical techniques based on information theoretic and algebraic approaches exist for performing ICA. In this paper, we have used an objective function based on independence criterion of the signals. Optimisation of this objective function yields a neural algorithm along with a non-linear function for signal separation. Performance of the algorithm for artificially generated signals as well as audio signals has been evaluated.     

基于判别式的核独立元分析*

核独立元分析（KICA）法是近年来发展起来的核化算法,但难以将其用于故障诊断问题。为了解决该问题,对两种核独立元分析算法——基于受限协方差测度的方法KICA2和基于核互信息测度的方法KICA3进行变形得到适用于分类或故障诊断的形式。进一步分析了KICA2与一种核偏最小二乘（KPLS）方法的等价性以及KICA3与核主元分析（KPCA）的等价性。最后对Tennessee Eastman过程进行仿真,说明了方法的有效性。     

Ensemble learning for independent component analysis

It is well known that the applicability of independent component analysis (ICA) to high-dimensional pattern recognition tasks such as face recognition often suffers from two problems. One is the small sample size problem. The other is the choice of basis functions (or independent components). Both problems make ICA classifier unstable and biased. In this paper, we propose an enhanced ICA algorithm by ensemble learning approach, named as random independent subspace (RIS), to deal with the two problems. Firstly, we use the random resampling technique to generate some low dimensional feature subspaces, and one classifier is constructed in each feature subspace. Then these classifiers are combined into an ensemble classifier using a final decision rule. Extensive experimentations performed on the FERET database suggest that the proposed method can improve the performance of ICA classifier.     

A novel process monitoring approach with dynamic independent component analysis

A novel process monitoring scheme is proposed to compensate for shortcomings in the conventional independent component analysis (ICA) based monitoring method. The primary idea is first to augment the observed data matrix in order to take the process dynamic into consideration. An outlier rejection rule is then proposed to screen out outliers, in order to better describe the majority of the data. Finally, a rectangular measure is used as a monitoring statistic. The proposed approach is investigated via three cases: a simulation example, the Tennessee Eastman process and a real industrial case. Results indicate that the proposed method is more efficient as compared to alternate methods.     

Algorithms for nonnegative independent component analysis

We consider the task of solving the independent component analysis (ICA) problem x=As given observations x, with a constraint of nonnegativity of the source random vector s. We refer to this as nonnegative independent component analysis and we consider methods for solving this task. For independent sources with nonzero probability density function (pdf) p(s) down to s=0 it is sufficient to find the orthonormal rotation y=Wz of prewhitened sources z=Vx, which minimizes the mean squared error of the reconstruction of z from the rectified version y/sup +/ of y. We suggest some algorithms which perform this, both based on a nonlinear principal component analysis (PCA) approach and on a geodesic search method driven by differential geometry considerations. We demonstrate the operation of these algorithms on an image separation problem, which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task.     

One-bit-matching conjecture for independent component analysis

The one-bit-matching conjecture for independent component analysis (ICA) could be understood from different perspectives but is basically stated as "all the sources can be separated as long as there is a one-to-one same-sign-correspondence between the kurtosis signs of all source probability density functions (pdf's) and the kurtosis signs of all model pdf's" (Xu, Cheung, & Amari, 1998a). This conjecture has been widely believed in the ICA community and implicitly supported by many ICA studies, such as the Extended Infomax (Lee, Girolami, & Sejnowski, 1999) and the soft switching algorithm (Welling & Weber, 2001). However, there is no mathematical proof to confirm the conjecture theoretically. In this article, only skewness and kurtosis are considered, and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes. Moreover, empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks. As a by-product, we also show that the kurtosis maximization criterion (Moreau & Macchi, 1996) is actually a special case of the minimum mutual information criterion for ICA.     

Principal independent component analysis

Conventional blind signal separation algorithms do not adopt any asymmetric information of the input sources, thus the convergence point of a single output is always unpredictable. However, in most of the applications, we are usually interested in only one or two of the source signals and prior information is almost always available. In this paper, a principal independent component analysis (PICA) concept is proposed. We try to extract the objective independent component directly without separating all the signals. A cumulant-based globally convergent algorithm is presented and simulation results are given to show the hopeful applicability of the PICA ideas.     

Least-squares independent component analysis

Accurately evaluating statistical independence among random variables is a key element of independent component analysis (ICA). In this letter, we employ a squared-loss variant of mutual information as an independence measure and give its estimation method. Our basic idea is to estimate the ratio of probability densities directly without going through density estimation, thereby avoiding the difficult task of density estimation. In this density ratio approach, a natural cross-validation procedure is available for hyperparameter selection. Thus, all tuning parameters such as the kernel width or the regularization parameter can be objectively optimized. This is an advantage over recently developed kernel-based independence measures and is a highly useful property in unsupervised learning problems such as ICA. Based on this novel independence measure, we develop an ICA algorithm, named least-squares independent component analysis.     

块独立成分分析的人脸识别

针对利用ICA提取人脸特征时需要将人脸图像转换成向量，导致空间维数很高以及不能准确地估计特征等问题，提出了一种新的独立子空间人脸识别算法——块独立成分分析（B-ICA）。和经典的ICA相比，B-ICA算法把人脸图像划分成一些互不重叠的子块，然后把每个子块转换成向量，看成是低维空间中的训练点（训练向量）。因此在B-ICA算法中，样本的维数比ICA算法中样本的维数低，降低了维数灾难（即样本的训练个数远小于样本的维数）造成的错误识别率。在Yale和AR数据库上进行了大量仿真实验，实验结果表明B-ICA算法的识别率比ICA和其他一些子空间算法的识别率高。     

Some global and local convergence analysis on the information-theoretic independent component analysis approach

In this paper, we present a detailed theoretical analysis on the information-theoretic Independent Component Analysis (IT-ICA) approach. We first provide a number of lemmas and theorems on properties of the corresponding cost function in the general n-channel case with differentiable, odd, monotonic decreasing nonlinearity. A theorem on behaviour of the cost function along a radially outward line is given for characterizing the global configuration of the cost function in the parameter space. Furthermore, on the 2-channel IT-ICA system with cubic nonlinearity, we not only exhaustively solve out all equilibrium points and the condition for stability, but also give a global convergence theorem.     

Topographic independent component analysis.

In ordinary independent component analysis, the components are assumed to be completely independent, and they do not necessarily have any meaningful order relationships. In practice, however, the estimated "independent" components are often not at all independent. We propose that this residual dependence structure could be used to define a topographic order for the components. In particular, a distance between two components could be defined using their higher-order correlations, and this distance could be used to create a topographic representation. Thus, we obtain a linear decomposition into approximately independent components, where the dependence of two components is approximated by the proximity of the components in the topographic representation.     

互信息配准的一种改进算法

在传统的互信息配准方法中,由于重叠面积的变化会引起配准测度的不稳定,导致误配准.本文提出了一种改进的方法,用"面积补偿"的原理来保证参考图像和待配准图像相对平移时重叠面积保持基本不变,有效地增加了配准测度在配准过程中的鲁棒性.通过对比实验证实了此方法的有效性.     

Investigation on the skewness for independent component analysis

Skewness has received much less attention than kurtosis in the independent component analysis (ICA).In particular,the skewness seems to become a useless statistics after the kurtosis related one-bit-matching theorem was proven.However,as the non-Gaussianity of one signal comes mainly from skewness,it is intuitively understandable that its recovery should not rely on kurtosis.In this paper we discuss the skewness based ICA,and show that any probability density function (pdf) with non-zero skewness can be emp...     

A unifying information-theoretic framework for independent component analysis

We show that different theories recently proposed for independent component analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra [1] and Cardoso [2] showed that the infomax approach of Bell and Sejnowski [3] and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties, and therefore, all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe [4] have shown that the nonlinear principal component analysis (PCA) algorithm of Karhunen and Joutsensalo [5] and Oja [6] can also be viewed from information-theoretic principles since it minimizes the sum of squares of the fourth-order marginal cumulants, and therefore, approximately minimizes the mutual information [7]. Lambert [8] has proposed different Bussgang cost functions for multichannel blind deconvolution. We show how the Bussgang property relates to the infomax principle. Finally, we discuss convergence and stability as well as future research issues in blind source separation.     

基于小波分解的独立分量掌纹识别方法

独立分量分析方法在图像处理中具有独特的优势，用于掌纹特征提取，使得变换后的各分量之间不仅互不相关，而且还尽可能的统计独立，能更全面的揭示掌纹特征间的本质结构。为了降低运算复杂度，提出了一种基于小波分解的独立分量掌纹特征提取方法。首先对掌纹图像做小波变换进行降维，在保留原始图像轮廓信息和细节信息的基础上，去掉高频噪声，然后进行独立分量分析，采用FastICA算法，试验结果表明,本方法比传统的独立分量分析方法的识别率更高，且计算量大大减少。     

参考独立分量分析固定点算法

参考独立分量分析将源信号的先验信息以参考信号的形式引入到算法中,仅实现期望源信号的抽取,消除了传统独立分量分析中抽取信号的不确定性;以期望信号和参考信号的接近性度量作为目标函数提出了一个固定点算法,避免了人为选取步长,同时通过优选初值进一步提高算法的收敛速率。针对合成数据和实际的心电图数据仿真实验,证明了算法的有效性和更好的收敛性。     

A constrained EM algorithm for independent component analysis

We introduce a novel way of performing independent component analysis using a constrained version of the expectation-maximization (EM) algorithm. The source distributions are modeled as D one-dimensional mixtures of gaussians. The observed data are modeled as linear mixtures of the sources with additive, isotropic noise. This generative model is fit to the data using constrained EM. The simpler "soft-switching" approach is introduced, which uses only one parameter to decide on the sub- or supergaussian nature of the sources. We explain how our approach relates to independent factor analysis.     

偏度在独立元分析模型中的作用分析及算法设计

相对于峭度(kurtosis),偏度(skewness)历来在独立元分析(ICA)的研究中就没有得到充分重视.尤其是当关于峭度符号的一比特匹配定理在理论上被证明了以后,偏度似乎更是变成了ICA模型中的一个无用统计量.但当信号的峭度很小或者其非Gauss性主要源自于偏度时,仅仅利用峭度信息是不足够的.本文目的就在于分析和讨论在此种情况下独立元分析如何利用偏度信息.首先从理论上分析了偏度在ICA模型中的作用,结果表明在偏度上并不存在与峭度类似的一比特匹配定理,也就是说,算法中模型密度函数的选择无需考虑其偏度与源信号偏度的符号匹配问题.在此基础上,本文进一步提出了一套灵活的模型密度函数设计方法,并提出了一个算法实例,它可以适用于具有任意偏度和峭度组合的信号.     

An independent component analysis filtering approach for estimating continental hydrology in the GRACE gravity data

An approach based on Independent Component Analysis (ICA) has been applied on a combination of monthly GRACE satellite solutions computed from official providers (CSR, JPL and GFZ), to separate useful geophysical signals from important striping undulations. We pre-filtered the raw GRACE Level-2 solutions using Gaussian filters of 300, 400, 500-km of radius to verify the non-Gaussianity condition which is necessary to apply the ICA. This linear inverse approach ensures to separate components of the observed gravity field which are statistically independent. The most energetic component found by ICA corresponds mainly to the contribution of continental water mass change. Series of ICA-estimated global maps of continental water storage have been produced over 08/2002-07/2009. Our ICA estimates were compared with the solutions obtained using other post-processing of GRACE Level-2 data, such as destriping and Gaussian filtering, at global and basin scales. Besides, they have been validated with in situ measurements in the Murray-Darling Basin. Our computed ICA grids are consistent with the different approaches. Moreover, the ICA-derived time series of water masses showed less north-south spurious gravity signals and improved filtering of unrealistic hydrological features at the basin-scale compared with solutions obtained using other filtering methods.