Discrimination with autocorrelated observations

The effect of serial correlation on the multivariate sample linear discriminant function when the parameters are estimated under the assumption that the observations are independent has been studied by Lawoko and McLachlan. This paper investigates the performance of the univariate sample linear discriminant function when the parameters are estimated under a model which takes the correlation into consideration. This is contrasted with the performance of the sample discriminant function when the parameters are estimated under the assumption that the observations are independent. The intraclass correlation model assumed here is the univariate autoregressive process of order p.     

基于双曲余弦矩阵鉴别分析的小样本问题研究

线性判别分析（LDA）是模式识别领域的一个经典方法,但是LDA难以克服小样本问题。针对LDA的小样本问题,提出一种双曲余弦矩阵鉴别分析方法（HCDA）。该方法首先给出了双曲余弦矩阵函数的定义及其特征系统,再利用双曲余弦矩阵函数特征系统的特点,将其引入Fisher准则中进行特征提取。HCDA有两方面的优势：a）避免了小样本问题,可以提取更多的鉴别信息;b）HCDA方法隐含了一个非线性映射。该映射具有扩大样本间距离的作用,并且对不同类别样本间距离的扩大尺度要大于同类别样本间距离的扩大尺度,从而更有利于模式分类。在手写数字库、手写字母图像库和Georgia Tech人脸图像库上的实验结果表明,相对于具有代表性的解决LDA小样本问题的方法,HCDA具有更好的识别性能。     

Further results on the effect of intraclass correlation among training samples in discriminant analysis

The effect of intraclass correlation among training samples on the misclasification probabilities of Bayes' procedure has been recently studied by Basu and Odell.⁽¹³⁾ This paper investigates the effect further by expressing the misclassification probabilities in the form of asymptotic expansions. It is subsequently shown that contrary to previous conclusions the misclassification probabilities do change in the presence of simple equicorrelation among the training samples.     

噪声方差不确定约束系统的滚动时域估计

针对噪声方差不确定的约束系统,讨论了一种鲁棒滚动时域估计(MHE)方法.首先,根据噪声方差不确定模型,找到满足所有不确定性的最小方差上界,在线性矩阵不等式(LMI)框架下求解优化问题,得到近似到达代价的表达形式;然后再融合预测控制的滚动优化原理,把系统的硬约束直接表述在优化问题中,在线优化性能指标,估计出当前时刻系统的状态.仿真时与鲁棒卡尔曼滤波方法进行比较,结果表明了该方法的有效性.     

Determination of the optimal number of features for quadratic discriminant analysis via the normal approximation to the discriminant distribution

Given the joint feature-label distribution, increasing the number of features always results in decreased classification error; however, this is not the case when a classifier is designed via a classification rule from sample data. Typically, for fixed sample size, the error of a designed classifier decreases and then increases as the number of features grows. The problem is especially acute when sample sizes are very small and the potential number of features is very large. To obtain a general understanding of the kinds of feature-set sizes that provide good performance for a particular classification rule, performance must be evaluated based on accurate error estimation, and hence a model-based setting for optimizing the number of features is needed. This paper treats quadratic discriminant analysis (QDA) in the case of unequal covariance matrices. For two normal class-conditional distributions, the QDA classifier is determined according to a discriminant. The standard plug-in rule estimates the discriminant from a feature-label sample to obtain an estimate of the discriminant by replacing the means and covariance matrices by their respective sample means and sample covariance matrices. The unbiasedness of these estimators assures good estimation for large samples, but not for small samples.Our goal is to find an essentially analytic method to produce an error curve as a function of the number of features so that the curve can be minimized to determine an optimal number of features. We use a normal approximation to the distribution of the estimated discriminant. Since the mean and variance of the estimated discriminant will be exact, these provide insight into how the covariance matrices affect the optimal number of features. We derive the mean and variance of the estimated discriminant and compare feature-size optimization using the normal approximation to the estimated discriminant with optimization obtained by simulating the true distribution of the estimated discriminant. Optimization via the normal approximation to the estimated discriminant provides huge computational savings in comparison to optimization via simulation of the true distribution. Feature-size optimization via the normal approximation is very accurate when the covariance matrices differ modestly. The optimal number of features based on the normal approximation will exceed the actual optimal number when there is large disagreement between the covariance matrices; however, this difference is not important because the true misclassification error using the number of features obtained from the normal approximation and the number obtained from the true distribution differ only slightly, even for significantly different covariance matrices.     

Error rate estimation on the basis of posterior probabilities

The so-called posterior probability estimator, e, formed by averaging the minimum of the posterior probabilities over a set of initial or additional observations (which need not be classified) is considered in the context of estimating the overall actual error rate for the linear discriminant function appropriate for two multivariate normal populations with a common covariance matrix. The bias of e is examined by deriving asymptotic approximations under three different models, the normal, logistic, and mixture models. The properties of e are investigated further by a series of simulation experiments for the logistic and mixture models for which there are few other available estimators.     

Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Classification based on Fisher's linear discriminant analysis (FLDA) is challenging when the number of variables largely exceeds the number of given samples. The original FLDA needs to be carefully modified and with high dimensionality implementation issues like reduction of storage costs are of crucial importance. Methods are reviewed for the high dimension/small sample size problem and the one closest, in some sense, to the classical regular approach is chosen. The implementation of this method with regard to computational and storage costs and numerical stability is improved. This is achieved through combining a variety of known and new implementation strategies. Experiments demonstrate the superiority, with respect to both overall costs and classification rates, of the resulting algorithm compared with other methods.     

Probabilistic fault diagnosis method based on the combination of nest-loop fisher discriminant analysis and analysis of relative changes

Bias of data location and increase in data variations are two typical disturbances, which in general, simultaneously exist in the fault process. Targeting their different characteristics, a nested-loop fisher discriminant analysis (NeLFDA) algorithm and relative changes (RC) algorithm are effectively combined for analyzing the fault characteristics. First, a prejudgment strategy is developed to evaluate the fault types and determine what changes are covered in the fault process. Two statistical indexes are defined, which conduct Monte Carlo based center fluctuation analysis and dissimilarity analysis respectively. Second, for the fault data containing those two faults simultaneously, a combined NeLFDA-RC algorithm is proposed for fault deviations modeling, which is termed as CNR-FD. Fault directions concerning bias of data location are extracted by the NeLFDA algorithm and then corresponding fault deviations are removed from the fault data. Then RC algorithm is performed on these fault data to extract directions concerning increase of data variations. These fault directions are used as reconstruction models to characterize each fault class. Particularly, the compromise between these two algorithms is determined by the Monte Carlo based center fluctuation analysis. For online applications, a probabilistic fault diagnosis strategy based on Bayes’ rule is performed to identify fault cause by discovering the right reconstruction models that can make the reconstructed monitoring statistics have the largest probabilities of belonging to normal condition. The motivation of the proposed algorithm is illustrated by a numerical case and the performance of the reconstruction models and the probabilistic fault diagnosis strategy are illustrated using pre-programmed faults from the Tennessee Eastman benchmark process and the real industrial process data from the cut-made process of cigarettes in some cigarette factory.     

Choice of B-splines with free parameters in the flexible discriminant analysis context

Flexible discriminant analysis (FDA) is a general methodology which aims at providing tools for multigroup non linear classification. It consists in a nonparametric version of discriminant analysis by replacing linear regression by any nonparametric regression method. A new option for FDA, consisting in a nonparametric regression method based on B-spline functions, will be introduced. The relevance of the transformation (hence the discrimination) depends on the parameters defining the spline functions: degree, number and location of the knots for each continuous variable. This method called FDA-FKBS (Free Knot B-Splines) allows to determine all these parameters without the necessity of many prior parameters. It is inspired by Reversible Jumps Monte Carlo Markov Chains but the objective function is different and the Bayesian aspect is put aside.     

A new approach to feature selection based on the Karhunen-Loeve expansion

After surveying existing feature selection procedures based upon the Karhunen-Loeve (K-L) expansion, the paper describes a new K-L technique that overcomes some of the limitations of the earlier procedures. The new method takes into account information on both the class variances and means, but lays particular emphasis on the classification potential of the latter. The results of a series of experiments concerned with the classification of real vector-electrocardiogram and artificially generated data demonstrate the advantages of the new method. They suggest that it is particularly useful for pattern recognition when combined with classification procedures based upon discriminant functions obtained by recursive least squares analysis.     

A delay decomposition approach to delay-dependent passivity analysis for interval neural networks with time-varying delay

This paper is concerned with delay-dependent passivity analysis for interval neural networks with time-varying delay. By decomposing the delay interval into multiple equidistant subintervals, new Lyapunov-Krasovskii functionals (LKFs) are constructed on these intervals. Employing these new LKFs, a new passivity criterion is proposed in terms of linear matrix inequalities, which is dependent on the size of the time delay. Finally, some numerical examples are given to illustrate the effectiveness of the developed techniques.     

An Approach to the Optimization of Solution of Problems of Training in Pattern Recognition on the Basis of the Method of Support Vectors

An approach is described to the optimization of solution of problems of training in pattern recognition on the basis of the method of support vectors with visualization of the results obtained.     

基于二进制可辨矩阵的肝功能检测数据约简方法

基于二进制可辨矩阵的数据约简方法具有直观性和可操作性的特点,因而在实际应用中受到开发人员的青睐。但已有的此类方法通常是在扫描数据集的过程中不加“削减”地直接产生大规模的二进制可辨矩阵,这导致较大的时间和空间开销。为此,利用对行的吸收律和逻辑和实现了对二进制可辨矩阵的规模进行有效缩减,构造一种新的基于二进制可辨矩阵的数据约简算法。它具有更好的可操作性,易于编程实现,其时间和空间复杂度都得到了较大的改善。在与某医院合作开发的项目中,该算法的应用已经进入测试阶段,可以较好完成了对肝功能检测数据（定性的数据）的约简,结果令人满意。     

Nonparametric maximum likelihood analysis of clustered current status data with the gamma-frailty Cox model

The Cox model with frailties has been popular for regression analysis of clustered event time data under right censoring. However, due to the lack of reliable computation algorithms, the frailty Cox model has been rarely applied to clustered current status data, where the clustered event times are subject to a special type of interval censoring such that we only observe for each event time whether it exceeds an examination (censoring) time or not. Motivated by the cataract dataset from a cross-sectional study, where bivariate current status data were observed for the occurrence of cataracts in the right and left eyes of each study subject, we develop a very efficient and stable computation algorithm for nonparametric maximum likelihood estimation of gamma-frailty Cox models with clustered current status data. The algorithm proposed is based on a set of self-consistency equations and the contraction principle. A convenient profile-likelihood approach is proposed for variance estimation. Simulation and real data analysis exhibit the nice performance of our proposal.     

结合PCANet与线性判别分析的视网膜光学相干断层扫描图像分类

目的 主成分分析网络（PCANet）能提取图像的纹理特征,线性判别分析（LDA）提取的特征有类别区分性。本文结合这两种方法的优点,提出一种带线性判别分析的主成分分析网络（PCANet-LDA）,用于视网膜光学相干断层扫描（OCT）图像中的老年性黄斑变性（AMD）、糖尿病性黄斑水肿（DME）及正常（NOR）这3类的全自动分类。方法 PCANet-LDA算法是在PCANet的基础上添加了LDA监督层,该层加入了类标签对特征进行监督投影。首先,对OCT视网膜图像进行去噪、二值化及对齐裁剪等一系列预处理,获得感兴趣的视网膜区域;然后,将预处理图像送入一个两层的PCA卷积层,训练PCA滤波器组并提取图像的PCA特征;接着,将PCA特征送入一个非线性输出层,通过二值散列和块直方图等处理,得到图像的特征;之后,将带有类标签的图像特征送入一个LDA监督层,学习LDA矩阵并用其对图像特征进行投影,使特征具有类别区分性;最后,将投影的特征送入线性支持向量机（SVM）中对分类器进行训练和分类。结果 实验分别在医院临床数据集和杜克数据集上进行,先对OCT图像预处理进行前后对比实验,然后对PCANet特征提取的有效性进行分析,最后对PCANet算法、ScSPM算法以及提出的PCANet-LDA3种分类算法的分类效果进行对比实验。在临床数据集上,PCANet-LDA算法的总体分类正确率为97.20%,高出PCANet算法3.77%,且略优于ScSPM算法;在杜克数据集上,PCANet-LDA算法的总体分类正确率为99.52%,高出PCANet算法1.64%,略优于ScSPM算法。结论 PCANet-LDA算法的分类正确率明显高于PCANet,且优于目前用于2D视网膜OCT图像分类的先进的ScSPM算法。因此,提出的PCANet-LDA算法在视网膜OCT图像的分类上是有效且先进的,可作为视网膜OCT图像分类的基准算法。     

多环芳烃(PAHs)对虹鳟(Oncorhynchus mykiss)鱼腮细胞系细胞毒性的Fisher判别分析

采用量子化学PM3算法计算得到的多环芳烃(PAHs)的量子化学参数,应用逐步判别分析法分析PAHs的细胞毒性,建立了能成功预测PAHs细胞毒性的Fisher线性判别函数,函数预测结果的正确识别率达到100％。研究认为影响PAHs细胞毒性的主要因素是PAHs的分子量(Mw)、偶极矩(μ)、分子最高占据轨道能(EHDMO)、分子最低未占据轨道能(ELUMO)和(ELUMO-EHOMO)^2。     

Stability analysis of a multi-model predictive control algorithm with application to control of chemical reactors

We study a stabilizing multi-model predictive control strategy for controlling nonlinear process at different operating conditions. The control algorithm is a receding horizon scheme with a quasi-infinite horizon objective function that has finite and infinite horizon cost components. The finite horizon cost consists of free input variables that direct the system towards a terminal region which contains the desired operating point. The infinite horizon cost has an upper bound and steers the system to the desired operating point. The system is represented by a sequence of piecewise linear models. Based on the condition of the system states, the sequence of piecewise linear models is updated and the controller’s objective function switches form quasi-infinite to infinite horizon objective function. This results in a hybrid control structure. A recent approach in the analysis of hybrid systems that uses multiple Lyapunov functions is employed in the stability analysis of the closed-loop system. The stabilizing hybrid control strategy is illustrated on two examples and their closed-loop stability properties are studied.