A constrained EM algorithm for independent component analysis

We introduce a novel way of performing independent component analysis using a constrained version of the expectation-maximization (EM) algorithm. The source distributions are modeled as D one-dimensional mixtures of gaussians. The observed data are modeled as linear mixtures of the sources with additive, isotropic noise. This generative model is fit to the data using constrained EM. The simpler "soft-switching" approach is introduced, which uses only one parameter to decide on the sub- or supergaussian nature of the sources. We explain how our approach relates to independent factor analysis.     

An adaptive learning algorithm for principal component analysis

Principal component analysis (PCA) is one of the most general purpose feature extraction methods. A variety of learning algorithms for PCA has been proposed. Many conventional algorithms, however, will either diverge or converge very slowly if learning rate parameters are not properly chosen. In this paper, an adaptive learning algorithm (ALA) for PCA is proposed. By adaptively selecting the learning rate parameters, we show that the m weight vectors in the ALA converge to the first m principle component vectors with almost the same rates. Comparing with the Sanger's generalized Hebbian algorithm (GHA), the ALA can quickly find the desired principal component vectors while the GHA fails to do so. Finally, simulation results are also included to illustrate the effectiveness of the ALA.     

A fast pixel mapping algorithm using principal component analysis

Pixel mapping is one of the basic processes in color quantization. In this paper, we shall propose a new algorithm using principal component analysis as a faster approach to pixel mapping. Within much shorter search time, our new scheme can find the nearest color which is identical to the one found using a full search. The idea behind the proposed method is quite simple. First, we compute two principal component directions (PCDs) for the palette. Then, the projected values on PCDs are computed for each color in palette. Finally, the projected values, following the triangular inequality principle, can help us reduce the computation time for finding the nearest color. The experimental results reveal that the proposed scheme is more efficient than the previous work.     

牛顿-软阈值迭代鲁棒主成分分析算法

针对鲁棒主成分分析（RPCA）问题,为了降低RPCA算法的时间复杂度,提出了牛顿-软阈值迭代（NSTI）算法。首先,使用低秩矩阵的Frobenius范数与稀疏矩阵的l₁-范数的和来构造NSTI算法的模型;其次,同时使用两种不同的优化方式求解模型的不同部分,即用牛顿法快速计算出低秩矩阵,用软阈值迭代算法快速计算出稀疏矩阵,交替使用这两种方法计算出原数据的低秩矩阵和稀疏矩阵的分解;最后,得到原始数据的低秩特征。在数据规模为5 000×5 000,低秩矩阵的秩为20的情况下,NSTI算法和梯度下降（GD）算法、低秩矩阵拟合（LMaFit）算法相比,时间效率分别提高了24.6%、45.5%。对180帧的视频前景背景进行分离,NSTI耗时3.63 s,时间效率比GD算法、LMaFit算法分别高78.7%、82.1%。图像降噪实验中,NSTI算法耗时0.244 s,所得到的降噪后的图像与原始图像的残差为0.381 3,与GD算法、LMaFit算法相比,时间效率和精确度分别提高了64.3%和45.3%。实验结果证明,NSTI算法能够有效解决RPCA问题并提升RPCA算法的时间效率。     

An iterative algorithm for robust kernel principal component analysis

We introduce a technique to improve iterative kernel principal component analysis (KPCA) robust to outliers due to undesirable artifacts such as noises, alignment errors, or occlusion. The proposed iterative robust KPCA (rKPCA) links the iterative updating and robust estimation of principal directions. It inherits good properties from these two ideas for reducing the time complexity, space complexity, and the influence of these outliers on estimating the principal directions. In the asymptotic stability analysis, we also show that our iterative rKPCA converges to the weighted kernel principal kernel components from the batch rKPCA. Experimental results are presented to confirm that our iterative rKPCA achieves the robustness as well as time saving better than batch KPCA.     

An efficient algorithm for Kernel two-dimensional principal component analysis

Recently, a new approach called two-dimensional principal component analysis (2DPCA) has been proposed for face representation and recognition. The essence of 2DPCA is that it computes the eigenvectors of the so-called image covariance matrix without matrix-to-vector conversion. Kernel principal component analysis (KPCA) is a non-linear generation of the popular principal component analysis via the Kernel trick. Similarly, the Kernelization of 2DPCA can be benefit to develop the non-linear structures in the input data. However, the standard K2DPCA always suffers from the computational problem for using the image matrix directly. In this paper, we propose an efficient algorithm to speed up the training procedure of K2DPCA. The results of experiments on face recognition show that the proposed algorithm can achieve much more computational efficiency and remarkably save the memory-consuming compared to the standard K2DPCA.     

牛顿-软阈值迭代鲁棒主成分分析算法

针对鲁棒主成分分析（RPCA）问题，为了降低RPCA算法的时间复杂度，提出了牛顿-软阈值迭代（NSTI）算法。首先，使用低秩矩阵的Frobenius范数与稀疏矩阵的l₁-范数的和来构造NSTI算法的模型；其次，同时使用两种不同的优化方式求解模型的不同部分，即用牛顿法快速计算出低秩矩阵，用软阈值迭代算法快速计算出稀疏矩阵，交替使用这两种方法计算出原数据的低秩矩阵和稀疏矩阵的分解；最后，得到原始数据的低秩特征。在数据规模为5 000×5 000，低秩矩阵的秩为20的情况下，NSTI算法和梯度下降（GD）算法、低秩矩阵拟合（LMaFit）算法相比，时间效率分别提高了24.6%、45.5%。对180帧的视频前景背景进行分离，NSTI耗时3.63 s，时间效率比GD算法、LMaFit算法分别高78.7%、82.1%。图像降噪实验中，NSTI算法耗时0.244 s，所得到的降噪后的图像与原始图像的残差为0.381 3，与GD算法、LMaFit算法相比，时间效率和精确度分别提高了64.3%和45.3%。实验结果证明，NSTI算法能够有效解决RPCA问题并提升RPCA算法的时间效率。     

一种基于贪心算法的快速PCA算法

提出一种快速算法,该算法利用贪心算法构造卷数据降维矩阵,在保持点与点之间“核距离”不变的情况下．把待分解矩阵变换成一个低维矩阵。在没有偏差的情况下,将对原始大矩阵的分解变成对这个低维矩阵的分解,大幅降低了时间复杂度,减少了对内存的使用率的同时增加了算法的稳定性。     

Grasshopper optimization algorithm with principal component analysis for global optimization

The Journal of Supercomputing - As one of the latest meta-heuristic algorithms, the grasshopper optimization algorithm (GOA) has extensive applications because of its efficiency and simplicity....     

A covariance-free iterative algorithm for distributed principal component analysis on vertically partitioned data

In this paper, a covariance-free iterative algorithm is developed to achieve distributed principal component analysis on high-dimensional data sets that are vertically partitioned. We have proved that our iterative algorithm converges monotonously with an exponential rate. Different from existing techniques that aim at approximating the global PCA, our covariance-free iterative distributed PCA (CIDPCA) algorithm can estimate the principal components directly without computing the sample covariance matrix. Therefore a significant reduction on transmission costs can be achieved. Furthermore, in comparison to existing distributed PCA techniques, CIDPCA can provide more accurate estimations of the principal components and classification results. We have demonstrated the superior performance of CIDPCA through the studies of multiple real-world data sets.     

A class of learning algorithms for principal component analysis andminor component analysis

In this paper, we first propose a differential equation for the generalized eigenvalue problem. We prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue. Based on this generalized differential equation, a class of principal component analysis (PCA) and minor component analysis (MCA) learning algorithms can be obtained. We demonstrate that many existing PCA and MCA learning algorithms are special cases of this class, and this class includes some new and simpler MCA learning algorithms. Our results show that all the learning algorithms of this class have the same order of convergence speed, and they are robust to implementation error.     

A class of learning algorithms for principal component analysis andminor component analysis

Principal component analysis (PCA) and minor component analysis (MCA) are a powerful methodology for a wide variety of applications such as pattern recognition and signal processing. In this paper, we first propose a differential equation for the generalized eigenvalue problem. We prove that the stable points of this differential equation are the eigenvectors corresponding to the largest eigenvalue. Based on this generalized differential equation, a class of PCA and MCA learning algorithms can be obtained. We demonstrate that many existing PCA and MCA learning algorithms are special cases of this class, and this class includes some new and simpler MCA learning algorithms. Our results show that all the learning algorithms of this class have the same order of convergence speed, and they are robust to implementation error.     

基于主分量分析的宽带DOA估计自聚焦算法

提出了一种基于主分量分析(PCA)的CSM类宽带DOA估计自聚焦算法,利用子空间投影变换将信号分离后应用PCA算法快速估计信号DOA,通过不断更新聚焦方向实现自聚焦.与已有算法相比,该算法不受DOA初始值的影响,有更好的聚焦精度.聚焦矩阵更新过程中无需再做奇异值分解,用PCA迭代算法替代特征分解过程,计算量小.仿真实验结果表明,该算法以较小的计算代价实现了较好的估计精度.     

基于主成分分析的超声人脸识别算法研究

针对超声波探测人脸识别系统中多通道探测模式,从数据融合的角度对特征进行了优化,研究了基于主成分分析(principal components analysis,PCA)的数据降维和人脸特征提取算法.利用该算法对100人的自由表情样本进行特征提取,在保证识别率超过80％前提下,可显著降低特征向量的维数达80％以上,提高系统速度85％以上.实验结果表明,PCA算法能有效降低特征数据的维数,提高运算速度.     

基于模糊理论决策的双向二维PCA 步态识别算法

针对步态识别中的平均步态能量图像系数矩阵维数过高和分类较困难的特 点,提出一种基于模糊理论决策分类的双向二维主成分分析的步态识别算法。通过预处理技 术得到平均步态能量图并将得到的图像分割为多个子图像,利用双向二维主成分分析来降低 平均步态能量子图像的系数矩阵维数,加快识别速度。引入模糊理论决策的方法进行最近邻 分类器的分类。最后在CASIA 步态数据库上对所提出的算法进行实验,实验结果表明该算 法具有较好的识别性能并有较强的鲁棒性。     

Accelerated k-nearest neighbors algorithm based on principal component analysis for text categorization

Text categorization is a significant technique to manage the surging text data on the Internet.The k-nearest neighbors(kNN) algorithm is an effective,but not efficient,classification model for text categorization.In this paper,we propose an effective strategy to accelerate the standard kNN,based on a simple principle:usually,near points in space are also near when they are projected into a direction,which means that distant points in the projection direction are also distant in the original space.Using the proposed strategy,most of the irrelevant points can be removed when searching for the k-nearest neighbors of a query point,which greatly decreases the computation cost.Experimental results show that the proposed strategy greatly improves the time performance of the standard kNN,with little degradation in accuracy.Specifically,it is superior in applications that have large and high-dimensional datasets.     

采用核聚类分析的KPCA改进算法

为了解决核主分量分析方法处理大训练样本集时计算代价巨大的问题,在采用子集划分的KPCA算法基础上,提出采用核聚类划分子集,并用每个子集的协方差矩阵的特征值累积贡献率作为标准来选取相应的特征向量.分别在人工和实际数据集上测试,实验结果显示在同一累积贡献率和给定子集个数的条件下,采用核聚类划分子集总能得到较小尺寸的核矩阵,而核矩阵尺寸的减小有助于改善测试样本的特征提取速度以及降低特征分解核矩阵的时间复杂度.     

基于鲁棒主成分分析的Canny边缘检测算法

为提高图像边缘检测的准确性和鲁棒性,提出一种基于鲁棒主成分分析(RPCA)的Canny边缘检测算法。该算法对图像进行RPCA分解得到图像的主成分和稀疏成分,利用Canny算子对主成分进行边缘检测,从而实现对图像的边缘检测。该算法将图像的边缘检测问题转化为图像主成分的边缘检测问题,消除了图像信息中“污点”对检测结果的干扰,抑制了噪声。仿真实验结果表明,该算法在边缘检测的准确性和鲁棒性方面优于Log边缘检测算法、Canny边缘检测算法和Susan边缘检测算法方法。     

A comparative analysis of principal component and independent component techniques for electrocardiograms

Principal component analysis (PCA) is used for ECG data compression, denoising and decorrelation of noisy and useful ECG components or signals. In this study, a comparative analysis of independent component analysis (ICA) and PCA for correction of ECG signals is carried out by removing noise and artifacts from various raw ECG data sets. PCA and ICA scatter plots of various chest and augmented ECG leads and their combinations are plotted to examine the varying orientations of the heart signal. In order to qualitatively illustrate the recovery of the shape of the ECG signals with high fidelity using ICA, corrected source signals and extracted independent components are plotted. In this analysis, it is also investigated if difference between the two kurtosis coefficients is positive than on each of the respective channels and if we get a super-Gaussian signal, or a sub-Gaussian signal. The efficacy of the combined PCA–ICA algorithm is verified on six channels V1, V3, V6, AF, AR and AL of 12-channel ECG data. ICA has been utilized for identifying and for removing noise and artifacts from the ECG signals. ECG signals are further corrected by using statistical measures after ICA processing. PCA scatter plots of various ECG leads give different orientations of the same heart information when considered for different combinations of leads by quadrant analysis. The PCA results have been also obtained for different combinations of ECG leads to find correlations between them and demonstrate that there is significant improvement in signal quality, i.e., signal-to-noise ratio is improved. In this paper, the noise sensitivity, specificity and accuracy of the PCA method is evaluated by examining the effect of noise, base-line wander and their combinations on the characteristics of ECG for classification of true and false peaks.     

Coupled principal component analysis

A framework for a class of coupled principal component learning rules is presented. In coupled rules, eigenvectors and eigenvalues of a covariance matrix are simultaneously estimated in coupled equations. Coupled rules can mitigate the stability-speed problem affecting noncoupled learning rules, since the convergence speed in all eigendirections of the Jacobian becomes widely independent of the eigenvalues of the covariance matrix. A number of coupled learning rule systems for principal component analysis, two of them new, is derived by applying Newton's method to an information criterion. The relations to other systems of this class, the adaptive learning algorithm (ALA), the robust recursive least squares algorithm (RRLSA), and a rule with explicit renormalization of the weight vector length, are established.