Boosting the partial least square algorithm for regression modelling

Boosting algorithms are a class of general methods used to improve the general performance of regression analysis. The main idea is to maintain a distribution over the train set. In order to use the given distribution directly, a modified PLS algorithm is proposed and used as the base learner to deal with the nonlinear multivariate regression problems. Experiments on gasoline octane number prediction demonstrate that boosting the modified PLS algorithm has better general performance over the PLS algorithm.     

Comparison of variable selection methods for PLS-based soft sensor modeling

Data-driven soft sensors have been widely used in both academic research and industrial applications for predicting hard-to-measure variables or replacing physical sensors to reduce cost. It has been shown that the performance of these data-driven soft sensors could be greatly improved by selecting only the vital variables that strongly affect the primary variables, rather than using all the available process variables. In this work, a comprehensive evaluation of different variable selection methods for PLS-based soft sensor development is presented, and a new metric is proposed to assess the performance of different variable selection methods. The following seven variable selection methods are compared: stepwise regression (SR), partial least squares with regression coefficients (PLS-BETA), PLS with variable importance in projection (PLS-VIP), uninformative variable elimination with PLS (UVE-PLS), genetic algorithm with PLS (GA-PLS), least absolute shrinkage and selection operator (Lasso), and competitive adaptive reweighted sampling with PLS (CARS-PLS). Their strengths and limitations for soft sensor development are demonstrated by a simulated case study and an industrial case study.     

Radial basis function networks, regression weights, and theexpectation-maximization algorithm

We propose a modified radial basis function (RBF) network in which the regression weights are used to replace the constant weights in the output layer. It is shown that the modified RBF network can reduce the number of hidden units significantly. A computationally efficient algorithm, known as the expectation-maximization (EM) algorithm, is used to estimate the parameters of the regression weights. A salient feature of this algorithm is that it decomposes a complicated multiparameter optimization problem into L separate small-scale optimization problems, where L is the number of hidden units. The superior performance of the modified RB network over the standard RBF network is illustrated by computer simulations     

Multi-way PLS regression: Monotony convergence of tri-linear PLS2 and optimality of parameters

The tri-linear PLS2 iterative procedure, an algorithm pertaining to the NIPALS framework, is considered. It was previously proposed as a first stage to estimate parameters of the multi-way PLS regression method. It is shown that the tri-linear PLS2 procedure is convergent. The procedure generates a sequence of parameters (scores and loadings), which can be described as increasing or decreasing two specific criteria. Furthermore, a hidden tensor is described allowing tri-linear PLS2 to search its best rank-one approximation. This tensor highlights the link between multi-way PLS regression and the well-known PARAFAC model. The parameters of the multi-way PLS regression method can be computed using three alternative procedures.     

Modified CC-LR algorithm with three diverse feature sets for motor imagery tasks classification in EEG based brain–computer interface

Motor imagery (MI) tasks classification provides an important basis for designing brain–computer interface (BCI) systems. If the MI tasks are reliably distinguished through identifying typical patterns in electroencephalography (EEG) data, a motor disabled people could communicate with a device by composing sequences of these mental states. In our earlier study, we developed a cross-correlation based logistic regression (CC-LR) algorithm for the classification of MI tasks for BCI applications, but its performance was not satisfactory. This study develops a modified version of the CC-LR algorithm exploring a suitable feature set that can improve the performance. The modified CC-LR algorithm uses the C3 electrode channel (in the international 10–20 system) as a reference channel for the cross-correlation (CC) technique and applies three diverse feature sets separately, as the input to the logistic regression (LR) classifier. The present algorithm investigates which feature set is the best to characterize the distribution of MI tasks based EEG data. This study also provides an insight into how to select a reference channel for the CC technique with EEG signals considering the anatomical structure of the human brain. The proposed algorithm is compared with eight of the most recently reported well-known methods including the BCI III Winner algorithm. The findings of this study indicate that the modified CC-LR algorithm has potential to improve the identification performance of MI tasks in BCI systems. The results demonstrate that the proposed technique provides a classification improvement over the existing methods tested.     

Improving the generalization performance of RBF neural networks using a linear regression technique

In this paper we present a method for improving the generalization performance of a radial basis function (RBF) neural network. The method uses a statistical linear regression technique which is based on the orthogonal least squares (OLS) algorithm. We first discuss a modified way to determine the center and width of the hidden layer neurons. Then, substituting a QR algorithm for the traditional Gram–Schmidt algorithm, we find the connected weight of the hidden layer neurons. Cross-validation is utilized to determine the stop training criterion. The generalization performance of the network is further improved using a bootstrap technique. Finally, the solution method is used to solve a simulation and a real problem. The results demonstrate the improved generalization performance of our algorithm over the existing methods.     

17集总动力学模型和线性PLS模型在连续重整装置芳烃收率在线预测上的应用

本论文将17集总反应动力学模型和线性回归模型用于在线预测工业级连续重整装置的芳烃收率。首先对17集总模型进行了简化处理从而提高模型计算速度,然后使用偏最小二乘法(PLS)对线性回归模型的参数进行估计,从而提高模型求解精度和稳定性。离线验证结果证明两个模型都是非常适合的。在此基础上,提出了有效的在线预测和在线校正策略,两个模型的在线预测精度分别是0.52wt%(重量百分数)和0.39wt%,与离线模拟精度几乎相当。两个模型可以同时用于在线预测,发挥各自的优势。     

基于多元最优信息规划的微粒群优化算法

在改进型微粒群优化算法的基础上，考虑到运算初期避免陷入局部寻优的需要，将多元最优信息分布状态知识引入对微粒群的总体运动规划，构造了适于求解高维多峰函数寻优问题的微粒群算法．仿真结果显示，该方法比改进型微粒群算法具有更好的总体寻优特性．     

A modified PLS path modeling algorithm handling reflective categorical variables and a new model building strategy

Partial least squares (PLS) path modeling has found increased applications in customer satisfaction analysis thanks to its ability to handle complex models. A modified PLS path modeling algorithm together with a model building strategy are introduced and applied to customer satisfaction analysis at the French energy supplier Electricité de France. The modified PLS algorithm handles all kinds of scales (categorical or nominal variables) and is well suited when nominal or binary variables are involved. PLS path modeling and structural equation modeling are confirmatory approaches and thus need an initial conceptual model. A two-step model building strategy is presented; the first step is based on Bayesian networks structure learning to build the measurement model and the second step is based on partial correlation and hypothesis tests to build the structural model. Applications to customer satisfaction data are presented.     

Local Dimensionality Reduction for Non-Parametric Regression

Locally-weighted regression is a computationally-efficient technique for non-linear regression. However, for high-dimensional data, this technique becomes numerically brittle and computationally too expensive if many local models need to be maintained simultaneously. Thus, local linear dimensionality reduction combined with locally-weighted regression seems to be a promising solution. In this context, we review linear dimensionality-reduction methods, compare their performance on non-parametric locally-linear regression, and discuss their ability to extend to incremental learning. The considered methods belong to the following three groups: (1) reducing dimensionality only on the input data, (2) modeling the joint input-output data distribution, and (3) optimizing the correlation between projection directions and output data. Group 1 contains principal component regression (PCR); group 2 contains principal component analysis (PCA) in joint input and output space, factor analysis, and probabilistic PCA; and group 3 contains reduced rank regression (RRR) and partial least squares (PLS) regression. Among the tested methods, only group 3 managed to achieve robust performance even for a non-optimal number of components (factors or projection directions). In contrast, group 1 and 2 failed for fewer components since these methods rely on the correct estimate of the true intrinsic dimensionality. In group 3, PLS is the only method for which a computationally-efficient incremental implementation exists. Thus, PLS appears to be ideally suited as a building block for a locally-weighted regressor in which projection directions are incrementally added on the fly.     

Soft sensor based on stacked auto-encoder deep neural network for air preheater rotor deformation prediction

Soft sensors have been widely used in industrial processes over the past two decades because they use easy-to-measure process variables to predict difficult-to-measure ones. Some success has been achieved by the dominant traditional methods of modeling soft sensors based on statistics, such as principal components analysis (PCA) and partial least square (PLS), but such sensors usually become inaccurate and inefficient when processing strong nonlinear data. In this paper, a new soft sensor modeling approach is proposed based on a deep learning network. First, stacked auto-encoders (SAEs) are employed to extract high-level feature representations of the input data. In the process of training each layer of a SAE, the Limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm (L-BFGS) is adopted to optimize the weights parameters. Then, a support vector regression (SVR) is added to predict the target value on the basis of the features obtained from the SAE. To improve the model performance, Genetic Algorithm (GA) is used to obtain the optimal parameters of the SVR. To evaluate the proposed method, a soft sensor model for estimating the rotor deformation of air preheaters in a thermal power plant boiler is studied. The experimental results demonstrate that the soft sensor based on the SAE-SVR algorithm is more effective than the existing methods are.     

GSM移动通信用户鉴权算法的分析与实现

首先分析研究了ＧＳＭ（Ｇｌｏｂａｌｓｙｓｔｅｍ ｆｏｒ ｍ ｏｂｉｌｅ）数字移动通信系统的安全技术,特别是有关用户鉴权与密钥分配的协议,然后以改进的ＦＥＡＬ算法为核心,构造了一个２Ｍ 比特并联杂凑ＦＥＡＬ－４（Ｘ）Ｓ加密算法,实现了ＧＳＭ 数字移动通信系统的身份认证与密钥分配的Ａ３／Ａ８ 算法。本文对该组加密算法及其性能进行了详细分析,并给出了在计算机上模拟的结果。     

An optimized instance based learning algorithm for estimation of compressive strength of concrete

This article proposes an optimized instance-based learning approach for prediction of the compressive strength of high performance concrete based on mix data, such as water to binder ratio, water content, super-plasticizer content, fly ash content, etc. The base algorithm used in this study is the k nearest neighbor algorithm, which is an instance-based machine leaning algorithm. Five different models were developed and analyzed to investigate the effects of the number of neighbors, the distance function and the attribute weights on the performance of the models. For each model a modified version of the differential evolution algorithm was used to find the optimal model parameters. Moreover, two different models based on generalized regression neural network and stepwise regressions were also developed. The performances of the models were evaluated using a set of high strength concrete mix data. The results of this study indicate that the optimized models outperform those derived from the standard k nearest neighbor algorithm, and that the proposed models have a better performance in comparison to generalized regression neural network, stepwise regression and modular neural networks models.     

Theoretical framework for local PLS1 regression, and application to a rainfall data set

A new multivariate non-parametric regression method is considered, which is an extension of PLS1 regression using the multivariate local polynomial regression framework. A theoretical framework that can be used in order to study the asymptotic properties of the estimator is proposed, and the method is implemented on a real data set, made up of seasonal amounts of rainfall in the north of the Nordeste region of Brazil, to be explained by climatic variables. The performances of the three methods of PLS1 regression, multivariate local regression and local PLS1 regression are compared by means of cross-validation and the use of a validation period. All calculations have been implemented in the S-Plus package. The results confirm the good properties of local PLS1 regression.     

基于累积量的混合信号识别算法性能分析

为提高存在定时误差条件下基于累积量特性的混合信号识别算法性能，本文给出了一种修正的混合信号识别算法。通过将混合信号模型代入累积量表达式，定量分析过采样率、定时偏差、基带成型脉冲函数对识别性能的影响，优化影响算法性能的关键参数实现途径，从而给出具有更高稳定性、更好识别率的修正算法。计算机仿真结果验证了修正算法的有效性。     

动态内全潜结构投影的空间扩展故障检测方法

动态内偏最小二乘(DiPLS)方法是基于数据驱动的潜结构投影的动态扩展算法, 用于动态特征提取和关键 性能指标预测. 在大型装备系统中, 传感器采集的当前时刻样本受历史样本的影响且可能包含较大噪声. 在动态特 征提取中, 因DiPLS算法未按降序提取主成分, 导致残差空间仍存在较大变异, 动态和静态信息难以有效分离, 影响 故障检测性能. 为此, 本文提出了一种基于动态内全潜结构投影的故障检测方法(DiTPLS). 首先, 使用动态内偏最小 二乘方法和向量自回归模型建立动态模型并检测故障, 用于捕捉质量相关动态信息; 基于结构化动态主成分分析 算法建立一种改进的动态潜在变量模型, 用于残差分解, 提取质量无关的动态信息和静态信息, 并构造合适的统计 量进行故障检测. 数值仿真和田纳西–伊斯曼过程实验验证了DiTPLS算法的有效性.     

核空间广义均衡模糊C-均值聚类算法

目的 针对现有广义均衡模糊C-均值聚类不收敛问题,提出一种改进广义均衡模糊聚类新算法,并将其推广至再生希尔伯特核空间以便提高该类算法的普适性。方法 在现有广义均衡模糊C-均值聚类目标函数的基础上,利用Schweizer T范数极限表达式的性质构造了新的广义均衡模糊C-均值聚类最优化目标函数,然后采用拉格朗日乘子法获取其迭代求解所对应的隶属度和聚类中心表达式,同时对其聚类中心迭代表达式进行修改并得到一类聚类性能显著改善的修正聚类算法;最后利用非线性函数将数据样本映射至高维特征空间获得核空间广义均衡模糊聚类算法。结果 对Iris标准文本数据聚类和灰度图像分割测试表明,提出的改进广义均衡模模糊聚类新算法及其修正算法具有良好的分类性能,核空间广义均衡模糊聚类算法对比现有融入类间距离的改进模糊C-均值聚类(FCS)算法和改进再生核空间的模糊局部C-均值聚类(KFLICM)算法能将图像分割的误分率降低10%30%。结论 本文算法克服了现有广义均衡模糊C-均值聚类算法的缺陷,同时改善了聚类性能,适合复杂数据聚类分析的需要。     

Partial least squares (PLS) based monitoring and control of batch digesters

In this paper, a data-based control method for reducing product quality variations in batch pulp digesters is presented. Compared to the existing techniques, the new technique uses more liquor measurements in predicting the final pulp quality. The liquor measurements obtained at different time instances during a cook are related to the final pulp quality through a partial least squares (PLS) regression model. In using the PLS regression model for control, two approaches are proposed. In the first approach, optimal control moves are computed directly using the PLS model, while the second approach employs a nonlinear H-factor model of which parameters are adapted using the prediction from the PLS model. The effectiveness of the prediction and control algorithms is examined through simulation studies. Experimental study is then performed on a lab-scale batch digester, to test the effectiveness of the prediction performance of the PLS model. The control algorithms will be tested on the experimental set-up in the future.     

Soft sensor design by multivariate fusion of image features and process measurements

This paper presents a multivariate data fusion procedure for design of dynamic soft sensors where suitably selected image features are combined with traditional process measurements to enhance the performance of data-driven soft sensors. A key issue of fusing multiple sensor data, i.e. to determine the weight of each regressor, is achieved through multivariate regression. The framework is described and illustrated with applications to cement kiln systems that are characterized by off-line quality measurements and on-line analyzers with limited reliability. Image features are extracted with a multivariate analysis technique from RGB pictures. The color information is also transformed to hue, saturation and intensity components. Both sets of image features are combined with traditional process measurements to obtain an inferential model by partial least squares (PLS) regression. A dynamic PLS model is obtained by filtering the original data block augmented with time lagged variables such that improved predictive performance of the quality variable results. Key issues regarding data preprocessing and extraction of suitable image features are discussed with a case study, the on-line estimation of nitrogen oxides (NOx) emission of cement kilns. On-site tests demonstrate improved performance over soft sensors based on conventional process measurements only.