Implementing projection pursuit learning

This paper examines the implementation of projection pursuit regression (PPR) in the context of machine learning and neural networks. We propose a parametric PPR with direct training which achieves improved training speed and accuracy when compared with nonparametric PPR. Analysis and simulations are done for heuristics to choose good initial projection directions. A comparison of a projection pursuit learning network with a single hidden-layer sigmoidal neural network shows why grouping hidden units in a projection pursuit learning network is useful. Learning robot arm inverse dynamics is used as an example problem.     

Fast initialization for cascade-correlation learning

Weight initialization in cascade-correlation learning is considered. Most of the previous studies use the so called candidate training to deal with the initialization problem in the cascade-correlation learning. There several candidate hidden units are first trained, and then the one yielding the best value for the covariance criterion is installed to the network. In case there are many candidate units to be trained, the total computational cost of the training can become very large. Here we consider a new approach for weight initialization in cascade-correlation learning. The proposed method is based on the concept of stepwise regression. Empirical simulations show that the new method can significantly speed-up cascade-correlation learning compared to the case where the candidate training is used. Moreover, the overall performance remained similar or was even better than with the candidate training.     

Modified cascade-correlation learning for classification

The main advantages of cascade-correlation learning are the abilities to learn quickly and to determine the network size. However, recent studies have shown that in many problems the generalization performance of a cascade-correlation trained network may not be quite optimal. Moreover, to reach a certain performance level, a larger network may be required than with other training methods. Recent advances in statistical learning theory emphasize the importance of a learning method to be able to learn optimal hyperplanes. This has led to advanced learning methods, which have demonstrated substantial performance improvements. Based on these recent advances in statistical learning theory, we introduce modifications to the standard cascade-correlation learning that take into account the optimal hyperplane constraints. Experimental results demonstrate that with modified cascade correlation, considerable performance gains are obtained compared to the standard cascade-correlation learning. This includes better generalization, smaller network size, and faster learning.     

Improving the approximation and convergence capabilities of projection pursuit learning

One nonparametric regression technique that has been successfully applied to high-dimensional data is projection pursuit regression (PPR). In this method, the regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors. Projection pursuit learning (PPL) proposed by Hwanget al. formulates PPR using a two-layer feedforward neural network. One of the main differences between PPR and PPL is that the smoothers in PPR are nonparametric, whereas those in PPL are based on Hermite functions of some predefined highest orderR. While the convergence property of PPR is already known, that for PPL has not been thoroughly studied. In this paper, we demonstrate that PPL networks do not have the universal approximation and strong convergence properties for any finiteR. But, by including a bias term in each linear combination of the predictor variables, PPL networks can regain these capabilities, independent of the exact choice ofR. It is also shown experimentally that this modification improves the generalization performance in regression problems, and creates smoother decision surfaces for classification problems.     

Connectionist projection pursuit regression

We present a novel regression method that combines projection pursuit regression with feed forward neural networks. The algorithm is presented and compared to standard neural network learning. Connectionist projection pursuit regression (CPPR) is applied to modelling the U.S. average dollar-Deutsch mark exchange rate movement using several economic indicators. The performance of CPPR is compared with the performances of other approaches to this problem.     

Use of bias term in projection pursuit learning improvesapproximation and convergence properties

In a regression problem, one is given a multidimensional random vector X, the components of which are called predictor variables, and a random variable, Y, called response. A regression surface describes a general relationship between X and Y. A nonparametric regression technique that has been successfully applied to high-dimensional data is projection pursuit regression (PPR). The regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors. Projection pursuit learning (PPL) formulates PPR using a 2-layer feedforward neural network. The smoothers in PPR are nonparametric, whereas those in PPL are based on Hermite functions of some predefined highest order R. We demonstrate that PPL networks in the original form do not have the universal approximation property for any finite R, and thus cannot converge to the desired function even with an arbitrarily large number of hidden units. But, by including a bias term in each linear projection of the predictor variables, PPL networks can regain these capabilities, independent of the exact choice of R. Experimentally, it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units, improves the generalization performance, and makes it less sensitive to the setting of R. Finally, we apply PPL to chaotic time series prediction, and obtain superior results compared with the cascade-correlation architecture.     

Combining topographic map formation with projection pursuit learning for nonparametric regression analysis

The original Self-Organizing Map (SOM) algorithm is known to perform poorly on regression problems due to the occurrence of nonfunctional mappings. Recently, we have introduced an unsupervised learning rule, called the Maximum Entropy learning Rule (MER), which performs topographic map formation without using a neighborhood function. In the present paper, MER is extended with a neighborhood function and applied to nonparametric projection pursuit regression. The extended rule, called eMER, alleviates the occurrence of nonfunctional mappings. The performance of our regression procedure is quantified and compared to other neural network-based parametric and nonparametric regression procedures.     

Application of projection pursuit learning to boundary detection and deblurring in images

Projection pursuit learning networks (PPLNs) have been used in many fields of research but have not been widely used in image processing. In this paper we demonstrate how this highly promising technique may be used to connect edges and produce continuous boundaries. We also propose the application of PPLN to deblurring a degraded image when little or no a priori information about the blur is available. The PPLN was successful at developing an inverse blur filter to enhance blurry images. Theory and background information on projection pursuit regression (PPR) and PPLN are also presented.     

Pattern classification using projection pursuit

This article discusses the adaptation of recently developed regression techniques to classifier design. Apart from finite sample effects, projection pursuit (PP) regression may be used to model a desired response (class) as a sum of ridge functions according to a minimum expected squared error criterion. This approach can be shown to furnish an optimal discriminant function which can satisfy the Neyman-Pearson criterion over all possible thresholds. Basis function expansions are used instead of smoothed histograms to reduce computation. Since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy, we introduce an improved method utilizing a nonlinear weighting function.     

A general exploratory projection pursuit network

Exploratory Projection Pursuit(EPP) is a statistical technique for finding interesting structure in high-dimensional data-sets. We introduce a negative feedback network which has been shown to perform EPP. We use the network with a novel but very simple activation function to search for different types of data structure where the form of the data structure is unspecified in advance.     

Gaussian mixture model classification: A projection pursuit approach

Gaussian mixture models (GMM) are commonly employed in nonparametric supervised classification. In high-dimensional problems it is often the case that information relevant to the separation of the classes is contained in a few directions. A GMM fitting procedure oriented to supervised classification is proposed, with the aim of reducing the number of free parameters. It resorts to projection pursuit as a dimension reduction method and combines it with GM modelling of class-conditional densities. In its derivation, issues regarding the forward and backward projection pursuit algorithms are discussed. The proposed procedure avoids the “curse of dimensionality”, is able to model structure in subspaces and regularizes the classification model. Its performance is illustrated on a simulation experiment and on a real data set, in comparison with other GMM-based classification methods.     

基于投影寻踪的Web软件复杂性度量

传统的软件复杂性度量方法主要是针对C/C++、Ada等语言开发的非Web应用。以面向对象的基于Struts框架的Web软件为研究对象,提出了适合于Web-Struts软件的3个方面的复杂性度量指标,并提出了一种基于带交叉算子人工鱼群和投影寻踪(PP)算法的Web应用软件复杂性度量方法。把Web软件多个复杂性度量指标综合成一维综合投影值,利用样本数据求解最佳投影方向,确定评价等级的综合投影值区间,根据测试样本综合投影值与区间值比较,获得综合评价结果。实例评价结果表明,所提方法具有较强的适用性和应用性。     

投影寻踪在瓦斯涌出量预测中的应用

针对瓦斯涌出量预测时常用的瓦斯监测数据降维方法会不同程度地造成数据信息丢失、导致预测精度降低的问题,利用投影寻踪原理并结合差分进化算法将高维样本数据转化成1维投影数据,运用Matlab曲线拟合工具箱建立了一种正弦和函数瓦斯涌出量预测模型。实验结果表明,该模型具有较高的预测精度和可操作性。     

Pair-perturbation influence functions of nongaussianity by projection pursuit

The most nongaussian direction to explore the clustering structure of the data is considered to be the interesting linear projection direction by applying projection pursuit. Nongaussianity is often measured by kurtosis, however, kurtosis is well known to be sensitive to influential points/outliers and the projection direction is essentially affected by unusual points. Hence in this paper we focus on developing the influence functions of projection directions to investigate the influence of abnormal observations especially on the pair-perturbation influence functions to uncover the masked unusual observations. A technique is proposed for defining and calculating influence functions for statistical functional of the multivariate distribution. A simulation study and a real data example are provided to illustrate the applications of these approaches.     

基于投影寻踪的无线传感器网络入侵检测模型

提出基于投影寻踪(PP)算法解决无线传感器网络入侵检测问题,利用PP算法将高维数据投影到低维数据空间,使得多特征属性的节点数据准确聚集.通过节点属性投影值的浮动来检测节点是否受到攻击.实验结果表明:基于PP的无线传感器网络入侵检测的方法在减少计算量,降低检测能耗的情况下,可以得到比传统的误差反向传播(BP)模型检测方法得到更好的检测效果.     

投影寻踪模型在软件质量评价中的应用

利用投影寻踪模型对软件质量多个指标综合成一维综合投影值,提出了基于遗传优化的投影寻踪模型在软件质量评价中的应用.利用样本数据求解最佳投影方向,确定评价等级的综合投影值区间,根据待评价样本综合投影值与区间值比较,得出综合评价.实例评价结果表明,投影寻踪模型应用于软件质量评价是可行的,具有较强的适用性和应用性,能够准确、科学和客观地评价软件质量.     

On the optimal solution to maximum margin projection pursuit

Multimedia Tools and Applications - Most existing dimensionality reduction methods have been applied as a separable data preprocessing step before classification algorithms. This reduces the...     

Generalized pursuit learning schemes: new families of continuous and discretized learning automata

The fastest learning automata (LA) algorithms currently available fall in the family of estimator algorithms introduced by Thathachar and Sastry (1986). The pioneering work of these authors was the pursuit algorithm, which pursues only the current estimated optimal action. If this action is not the one with the minimum penalty probability, this algorithm pursues a wrong action. In this paper, we argue that a pursuit scheme that generalizes the traditional pursuit algorithm by pursuing all the actions with higher reward estimates than the chosen action, minimizes the probability of pursuing a wrong action, and is a faster converging scheme. To attest this, we present two new generalized pursuit algorithms (GPAs) and also present a quantitative comparison of their performance against the existing pursuit algorithms. Empirically, the algorithms proposed here are among the fastest reported LA to date.     

A projection pursuit algorithm for anomaly detection in hyperspectral imagery

The main goal of this paper is to propose an innovative technique for anomaly detection in hyperspectral imageries. This technique allows anomalies to be identified whose signatures are spectrally distinct from their surroundings, without any a priori knowledge of the target spectral signature. It is based on an one-dimensional projection pursuit with the Legendre index as the measure of interest. The index optimization is performed with a simulated annealing over a simplex in order to bypass local optima which could be sub-optimal in certain cases. It is argued that the proposed technique could be considered as seeking a projection to depart from the normal distribution, and unfolding the outliers as a consequence. The algorithm is tested with AHS and HYDICE hyperspectral imageries, where the results show the benefits of the approach in detecting a great variety of objects whose spectral signatures have sufficient deviation from the background. The technique proves to be automatic in the sense that there is no need for parameter tuning, giving meaningful results in all cases. Even objects of sub-pixel size, which cannot be made out by the human naked eye in the original image, can be detected as anomalies. Furthermore, a comparison between the proposed approach and the popular RX technique is given. The former outperforms the latter demonstrating its ability to reduce the proportion of false alarms.     

量子粒子群在投影寻踪聚类中的应用

提出基于量子粒子群的投影寻踪聚类算法,该算法将量子粒子群的全局搜索能力与投影寻踪对高维数据的降维能力相结合,有效解决了高维数据聚类计算量大效率低的问题。并将该算法应用于三种不同的测试数据,仿真实验结果表明该算法具有更好的效率,且提高了聚类效果,是解决高维聚类问题的一种有效方法。