Determination of state-space model uncertainty using bootstrap techniques

Robust control theory is widely used as the theoretical basis for the design of controllers with reduced sensibility to model errors. The model parameters variance–covariance (VC) matrix allows to design controllers with a consistent control action, even in the presence of moderate model mismatch. This paper presents a technique to extract the state-space model variance–covariance matrix using bootstrap techniques. The VC matrix is estimated from bootstrapped models using a first-order approximation of the model parameters space. The technique is applied by estimating the nominal model uncertainty of a deisopentanizer petrochemical unit. The model uncertainty is determined more accurately by the proposed method, when compared to the use of minimal canonical parameterization, providing better first-order approximation confidence intervals.     

A covariance-free iterative algorithm for distributed principal component analysis on vertically partitioned data

In this paper, a covariance-free iterative algorithm is developed to achieve distributed principal component analysis on high-dimensional data sets that are vertically partitioned. We have proved that our iterative algorithm converges monotonously with an exponential rate. Different from existing techniques that aim at approximating the global PCA, our covariance-free iterative distributed PCA (CIDPCA) algorithm can estimate the principal components directly without computing the sample covariance matrix. Therefore a significant reduction on transmission costs can be achieved. Furthermore, in comparison to existing distributed PCA techniques, CIDPCA can provide more accurate estimations of the principal components and classification results. We have demonstrated the superior performance of CIDPCA through the studies of multiple real-world data sets.     

An improved weighted recursive PCA algorithm for adaptive fault detection

A novel weighted adaptive recursive fault detection technique based on Principal Component Analysis (PCA) is proposed to address the issue of the increment in false alarm rate in process monitoring schemes due to the natural, slow and normal process changes (aging), which often occurs in real processes. It has been named as weighted adaptive recursive PCA (WARP).The aforementioned problem is addressed recursively by updating the eigenstructure (eigenvalues and eigenvectors) of the statistical detection model when the false alarm rate increases given the awareness of non-faulty condition. The update is carried out by incorporating the new available information within a specific online process dataset, instead of keeping a fixed statistical model such as conventional PCA does. To achieve this recursive updating, equations for means, standard deviations, covariance matrix, eigenvalues and eigenvectors are developed. The statistical thresholds and the number of principal components are updated as well.A comparison between the proposed algorithm and other recursive PCA-based algorithms is carried out in terms of false alarm rate, misdetection rate, detection delay and its computational complexity. WARP features a significant reduction of the computational complexity while maintaining a similar performance on false alarm rate, misdetection rate and detection delay compared to that of the other existing PCA-based recursive algorithms. The computational complexity is assessed in terms of the Floating Operation Points (FLOPs) needed to carry out the update.     

Principal component analysis with optimum order sample correlation coefficient for image enhancement

Principal component analysis (PCA) has been commonly used and has played an important role in remote sensing for information extraction. However, the ordinary PCA based on second‐order covariance or correlation is capable of forming components on the basis of the statistical properties of a majority of pixel values – pixel values around mean values. For many applications, principal components should be constructed on the basis of optimum correlation coefficients so that the components can represent low or high values of minority pixels of interest. A new version of the PCA has been proposed on the basis of an optimum order sample correlation coefficient for enhancing the contribution of the image bands including the low or high minority pixel values that can assist in extracting weak information for image classification and pattern recognition. The ordinary PCA becomes the special case of the new version of the PCA introduced in this paper. The new method was validated with a case study of identification of Au/Cu‐associated alteration zones from a Landsat Thematic Mapper (TM) image in the Mitchell‐Sulphurets district, Canada.     

基于故障检测率的主元个数确定方法

主元个数是PCA模型的关键参数,其选取直接决定PCA的故障诊断性能;针对传统主元个数选取方法主观性较大,且不考虑故障诊断要求的缺点,提出一种改进的主元个数确定方法;该方法将传统的累积方差贡献率与故障检测率相结合,首先利用累积方差贡献率初步确定主元个数,然后确定满足故障检测率要求的主元个数,将两个主元个数进行比较,从而获得最佳主元个数;与单纯累积方差贡献率方法相比,提高了主元模型的精度,减少了以往方法中人为因素的影响;通过对卫星控制系统的故障检测,证实了该方法可大大提高故障检测准确率。     

On the relationships between SVD, KLT and PCA

In recent literature on digital image processing much attention is devoted to the singular value decomposition (SVD) of a matrix. Many authors refer to the Karhunen-Loeve transform (KLT) and principal components analysis (PCA) while treating the SVD. In this paper we give definitions of the three transforms and investigate their relationships. It is shown that in the context of multivariate statistical analysis and statistical pattern recognition the three transforms are very similar if a specific estimate of the column covariance matrix is used. In the context of two-dimensional image processing this similarity still holds if one single matrix is considered. In that approach the use of the names KLT and PCA is rather inappropriate and confusing. If the matrix is considered to be a realization of a two-dimensional random process, the SVD and the two statistically defined transforms differ substantially.     

A G&P EWMA algorithm for high-mix semiconductor manufacturing processes

In mixed run processes, typical in semiconductor manufacturing and other automated assembly-line type process, products with different recipes will be produced on the same tool. Product based run-to-run control can be applied to improve the process capability. The effect of product-based controller on low frequency products is, however, minimal, due to inability to track tool variations. In this work, we propose a group and product based EWMA control scheme which combines adaptive k-means cluster method and run-to-run EWMA control to improve the performance of low frequency products in the mixed run process. Similar products could be classified into the same group adaptively and controlled by a group EWMA controller. The group controller is updated by both low frequency products and similar high frequency products; so that low frequency products can be improved by shared information from similar large frequency products. However, the high frequency products are controlled by individual product-based EWMA to avoid interference of the low frequency products. The advantages of proposed control scheme are demonstrated by benchmark simulation and reversed engineered industrial applications.     

A new and practical influence measure for subsets of covariance matrix sample principal components with applications to high dimensional datasets

Principal Component Analysis (PCA) is an important tool in multivariate analysis, in particular when faced with high dimensional data. There has been much done with regard to sensitivity analysis and the development of influence diagnostics for the eigenvector estimators that define the sample principal components. However, little, if any, has been done in this setting with regard to the sample principal components themselves. In this paper we develop a sensitivity measure for principal components associated with the covariance matrix that is very much related to the influence function (Hampel, 1974). This influence measure is based on the average squared canonical correlation and differs from the existing measures in that it assesses the influence of certain observational types on the sample principal components. We use this measure to derive an influence diagnostic that satisfies two key criteria being (i) it detects influential observations with respect to subsets of sample principal components and (ii) is efficient to calculate even in high dimensions. We use several microarray datasets to show that our measure satisfies both criteria.     

A variable selection method in principal canonical correlation analysis

We propose a variable selection procedure for the canonical correlation analysis (CCA) between two sets of principal components. We attempt to create predictive models for selecting such variables by combining principal component analysis (PCA) and CCA, and we refer to them collectively as principal canonical correlation analysis (PCCA). We derive a model selection criterion of one set of principal components, based on the selection of a covariance structure analysis within the framework of the PCCA. Compared to the variable selection procedure used in the CCA, the procedure used in the PCCA return a smaller number of variables. This is because the principal components derived from a PCA descend in order of the amount of information that they contain. The principal components with the smallest variance contributions are disregarded because their information contribution becomes negligible. Herein, we demonstrate the effectiveness of this criterion by using an example. Moreover, we investigate the properties of a variable selection criterion using the bootstrap resampling. The variable selection procedure used with the PCCA is compared to that used for the CCA.     

Principal components selection for dimensionality reduction using discriminant information applied to fault diagnosis

The Principal Component Analysis is one of most applied dimensionality reduction techniques for process monitoring and fault diagnosis in industrial process. This work proposes a procedure based on the discriminant information contained in the principal components to determine the most significant ones in fault separability. The Tennessee Eastman Process industrial benchmark is used to illustrate the effectiveness of the proposal. The use of statistical hypothesis tests as a separability measure between multiple failures is proposed for the selection of the principal components. The classifier profile concept has been introduced for comparison purposes. Results show an improvement in the classification process when compared with traditional techniques and the StepWise selection. This has resulted in a better classification for a fixed number of components, or a smaller number of required components to obtain a prefixed error rate. In addition, the computational advantage is demonstrated.     

基于稀疏主元分析的过程监控研究

主元分析（principal component analysis）是一种多元统计技术,在过程监控和故障诊断中具有广泛的应用。针对过程监控中数据量大的特点,提出一种稀疏主元分析（sparse principal component analysis）方法,通过引入lasso约束函数,构建稀疏主元分析的框架,将PCA降维问题转化为回归最优化问题,从而求解得到稀疏化的主元,并提高了主元模型的抗干扰能力。由于稀疏后主元相关的数据量减少,利用数据建立过程监控模型,减少了计算量,并缩短了计算时间,进而提高了监控的实时性。利用田纳西伊斯特曼过程（TE processes）进行实验仿真,并与传统的主元分析方法进行对比研究。结果表明,新提出的稀疏主元分析方法在计算效率和监控实时性上均优于传统的主元分析方法。     

A new ranking method for principal components analysis and its application to face image analysis

In this work, we investigate a new ranking method for principal component analysis (PCA). Instead of sorting the principal components in decreasing order of the corresponding eigenvalues, we propose the idea of using the discriminant weights given by separating hyperplanes to select among the principal components the most discriminant ones. The method is not restricted to any particular probability density function of the sample groups because it can be based on either a parametric or non-parametric separating hyperplane approach. In addition, the number of meaningful discriminant directions is not limited to the number of groups, providing additional information to understand group differences extracted from high-dimensional problems. To evaluate the discriminant principal components, separation tasks have been performed using face images and three different databases. Our experimental results have shown that the principal components selected by the separating hyperplanes allow robust reconstruction and interpretation of the data, as well as higher recognition rates using less linear features in situations where the differences between the sample groups are subtle and consequently most difficult for the standard and state-of-the-art PCA selection methods.     

Dimensionality reduction for multivariate time-series data mining

A multivariate time series is one of the most important objects of research in data mining. Time and variables are two of its distinctive characteristics that add the complication of the algorithms applied to data mining. Reduction in the dimensionality is often regarded as an effective way to address these issues. In this paper, we propose a method based on principal component analysis (PCA) to effectively reduce the dimensionality. We call it “piecewise representation based on PCA” (PPCA), which segments multivariate time series into several sequences, calculates the covariance matrix for each of them in terms of the variables, and employs PCA to obtain the principal components in an average covariance matrix. The results of the experiments, including retained information analysis, classification, and a comparison of the central processing unit time consumption, demonstrate that the PPCA method used to reduce the dimensionality in multivariate time series is superior to the prior methods.

     

Low complexity adaptive algorithms for Principal and Minor Component Analysis

This article introduces new low cost algorithms for the adaptive estimation and tracking of principal and minor components. The proposed algorithms are based on the well-known OPAST method which is adapted and extended in order to achieve the desired MCA or PCA (Minor or Principal Component Analysis). For the PCA case, we propose efficient solutions using Givens rotations to estimate the principal components out of the weight matrix given by OPAST method. These solutions are then extended to the MCA case by using a transformed data covariance matrix in such a way the desired minor components are obtained from the PCA of the new (transformed) matrix. Finally, as a byproduct of our PCA algorithm, we propose a fast adaptive algorithm for data whitening that is shown to overcome the recently proposed RLS-based whitening method.     

A multi-scale orthogonal nonlinear strategy for multi-variate statistical process monitoring

In this paper a multi-scale nonlinear PCA strategy for process monitoring is proposed. The strategy utilizes the optimal wavelet decomposition in such a way that only the approximation and the highest detail functions are used, thus simplifying the overall structure and making the interpretation at each scale more meaningful. An orthogonal nonlinear PCA procedure is incorporated to capture the nonlinear characteristics with a minimum number of principal components. The proposed nonlinear strategy also eliminates the requirement of nonlinear functions relating the nonlinear principal scores to process measurements for Q-statistics as in other nonlinear PCA process monitoring approaches. In addition, the strategy is considerably robust to the presence of typical outliers.     

基于主元分析和小波变换的开工过程故障识别

针对蒸汽裂解实验装置的开工过程具有间歇操作,变量间相关性高的特点,传统的故障识别方法无法有效处理这种具有较强动态特性的实际工业生产过程.本文提出利用主元分析,用少量主元反映多个动态变量的综合信息,并利用正交小波变换的多尺度时频分析提取主元中表征工况变化的频带特征,对频带特征进行模式归纳分类,进而识别工况.实验结果证实了所提出方法的可行性和有效性.     

Image covariance-based subspace method for face recognition

This paper proposes a new subspace method that is based on image covariance obtained from windowed features of images. A windowed input feature consists of a number of pixels, and the dimension of input space is determined by the number of windowed features. Each element of an image covariance matrix can be obtained from the inner product of two windowed features. The 2D-PCA and 2D-LDA methods are then obtained from principal component analysis and linear discriminant analysis, respectively, using the image covariance matrix. In the case of 2D-LDA, there is no need for PCA preprocessing and the dimension of subspace can be greater than the number of classes because the within-class and between-class image covariance matrices have full ranks. Comparative experiments are performed using the FERET, CMU, and ORL databases of facial images. The experimental results show that the proposed 2D-LDA provides the best recognition rate among several subspace methods in all of the tests.     

Asynchronism-based principal component analysis for time series data mining

Principal component analysis (PCA) is often applied to dimensionality reduction for time series data mining. However, the principle of PCA is based on the synchronous covariance, which is not very effective in some cases. In this paper, an asynchronism-based principal component analysis (APCA) is proposed to reduce the dimensionality of univariate time series. In the process of APCA, an asynchronous method based on dynamic time warping (DTW) is developed to obtain the interpolated time series which derive from the original ones. The correlation coefficient or covariance between the interpolated time series represents the correlation between the original ones. In this way, a novel and valid principal component analysis based on the asynchronous covariance is achieved to reduce the dimensionality. The results of several experiments demonstrate that the proposed approach APCA outperforms PCA for dimensionality reduction in the field of time series data mining.     

A novel model-based approach to optimization and control design of batch processes

Traditionally, many industrial batch processes have been operated according to rigid recipies, in spite of the fact that production would yield more profit or a better product if they were efficiently adapted to changes in quality and cost of the used and/or produced products, process and scheduling conditions, and so on. In this paper a approach, called the flexible recipe approach, is given, which transforms the common rather static recipes into recipes that can be easily improved and used for systematic and efficient production adaptation at the start of a batch and during the processing. To be able to use this approach in an industrial environment a practical implementation is made in the software package FRIS. A fermentation process chosen as an example shows the methods and gives an indication of the expected profit.