Model-based mixture discriminant analysis—an experimental study

The subject of this paper is an experimental study of a discriminant analysis (DA) based on Gaussian mixture estimation of the class-conditional densities. Five parameterizations of the covariance matrixes of the Gaussian components are studied. Recommendation for selection of the suitable parameterization of the covariance matrixes is given.     

High-order Fisher's discriminant analysis

This paper introduces a novel nonlinear extension of Fisher's classical linear discriminant analysis (FDA) known as high-order Fisher's discriminant analysis (HOFDA). The ability of the new method to capture nonlinear relationships stems from its use of an extended polynomial space constructed out of the original features. Furthermore, a genetic algorithm (GA) is used in order to incrementally generate an optimal subset of polynomial features out of an initial pool of minimal discriminants. This procedure yields surprisingly compact discriminants with state of the art recognition rates for the difficult UCI thyroid classification problem.     

基于模型的软件安全预测与分析

为了有效表示和分析软件中存在的安全缺陷和隐患,基于模型的软件安全分析技术采用多层次建模技术实现安全特征的描述,在评价软件及软件组件间安全性的过程中提出软件安全预测技术.通过分析软件组成成分之间的关联度获得相关的安全距离,在此基础之上生成安全依赖图,最后根据安全依赖图进行安全预测和分析.基于模型的安全分析技术能够针对可能存在的安全隐患给出预测和分析,为软件的测试和维护提供依据和手段.     

Genetic algorithms combined with discriminant analysis for key variable identification

Many trouble-shooting problems in process industries are related to key variable identification for classifications. The contribution charts, based on principal component analysis (PCA), can be applied for this purpose. Genetic algorithms (GAs) have been proposed recently for many applications including variable selection for multivariate calibration, molecular modeling, regression analysis, model identification, curve fitting, and classification. In this paper, GAs are incorporated with Fisher discriminant analysis (FDA) for key variable identification. GAs are used as an optimization tool to determine variables that maximize the FDA classification success rate for two given data sets. GA/FDA is a proposed solution for the variable selection problem in discriminant analysis. The Tennessee Eastman process (TEP) simulator was used to generate the data sets to evaluate the correctness of the key variable selection using GA/FDA, and the T² and Q statistic contribution charts. GA/FDA correctly identifies the key variables for the TEP case studies that were tested. For one case study where the correlation changes in two data sets, the contribution charts incorrectly suggest that the operating conditions are similar. On the other hand, GA/FDA not only determines that the operating conditions are different, but also identifies the key variables for the change. For another case study where many key variables are responsible for the changes in the two data sets, the contribution charts only identifies a fraction of the key variables, while GA/FDA correctly identifies all of the key variables. GA/FDA is a promising technique for key variable identification, as is evidenced in successful applications at The Dow Chemical Company.     

Bayesian analysis of the patterns of biological susceptibility via reversible jump MCMC sampling

In some biological experiments, it is quite common that laboratory subjects differ in their patterns of susceptibility to a treatment. Finite mixture models are useful in those situations. In this paper we model the number of components and the component parameters jointly, and base inference about these quantities on their posterior probabilities, making use of the reversible jump Markov chain Monte Carlo methods. In particular, we apply the methodology to the analysis of univariate normal mixtures with multidimensional parameters, using a hierarchical prior model that allows weak priors while avoiding improper priors in the mixture context. The practical significance of the proposed method is illustrated with a dose-response data set.     

Model-based testing for real

Model-based testing relies on abstract behavior models for test case generation. These models are abstractions, i.e., simplifications. For deterministic reactive systems, test cases are sequences of input and expected output. To bridge the different levels of abstraction, input must be concretized before being applied to the system under test. The systems output must then be abstracted before being compared to the output of the model.The concepts are discussed along the lines of a feasibility study, an inhouse smart card case study. We describe the modeling concepts of the CASE tool AutoFocus and an approach to model-based test case generation that is based on symbolic execution with Constraint Logic Programming.Different search strategies and algorithms for test case generation are discussed. Besides validating the model itself, generated test cases were used to verify the actual hardware with respect to these traces.     

Unified formulation of linear discriminant analysis methods and optimal parameter selection

In the last decade, many variants of classical linear discriminant analysis (LDA) have been developed to tackle the under-sampled problem in face recognition. However, choosing the variants is not easy since these methods involve eigenvalue decomposition that makes cross-validation computationally expensive. In this paper, we propose to solve this problem by unifying these LDA variants in one framework: principal component analysis (PCA) plus constrained ridge regression (CRR). In CRR, one selects the target (also called class indicator) for each class, and finds a projection to locate the class centers at their class targets and the transform minimizes the within-class distances with a penalty on the transform norm as in ridge regression. Under this framework, many existing LDA methods can be viewed as PCA+CRR with particular regularization numbers and class indicators and we propose to choose the best LDA method as choosing the best member from the CRR family. The latter can be done by comparing their leave-one-out (LOO) errors and we present an efficient algorithm, which requires similar computations to the training process of CRR, to evaluate the LOO errors. Experiments on Yale Face B, Extended Yale B and CMU-PIE databases are conducted to demonstrate the effectiveness of the proposed methods.     

基于元认知模型的智能混合高斯背景建模

传统混合高斯背景模型(Gaussian mixture model, GMM)不能快速适应动态场景中背景发生突变的情况. 本文提出一种基于元认知模型的智能混合高斯背景建模方法, 每个输入像素经过元认知监控成分刺激元认知体验成分以提取成功(或失败)的意识进行认知, 根据提取的意识及时向元认知知识成分传输新的认知知识或直接提取元认知知识成分, 并作出决策信息. 该方法能够对背景模型产生认知, 当背景突变为认知过的背景时, 可以快速适应并能更准确地描述复杂场景中的真实背景.     

Independent factor discriminant analysis

In the general classification context the recourse to the so-called Bayes decision rule requires to estimate the class conditional probability density functions. A mixture model for the observed variables which is derived by assuming that the data have been generated by an independent factor model is proposed. Independent factor analysis is in fact a generative latent variable model whose structure closely resembles the one of the ordinary factor model, but it assumes that the latent variables are mutually independent and not necessarily Gaussian. The method therefore provides a dimension reduction together with a semiparametric estimate of the class conditional probability density functions. This density approximation is plugged into the classic Bayes rule and its performance is evaluated both on real and simulated data.     

Model-based resource analysis and synthesis of service-oriented automotive software architectures

Software and Systems Modeling - Automotive software architectures describe distributed functionality by an interaction of software components. One drawback of today’s architectures is their...     

Classification into homogeneous groups using combined cluster and discriminant analysis

The classification of observations into groups is a general procedure in modern research. However, when searching for homogeneous groups the difficulty of deciding whether further division of a classification is necessary or not to obtain the desired homogeneous groups arises. The presented method, Combined cluster and discriminant analysis (CCDA), aims to facilitate this decision.CCDA consists of three main steps: (I) a basic grouping procedure; (II) a core cycle where the goodness of preconceived and random classifications is determined; and (III) an evaluation step where a decision has to be made regarding division into sub-groups. These steps of the proposed method were implemented in R in a package, under the name of ccda.To present the applicability of the method, a case study on the water quality samples of Neusiedler See is presented, in which CCDA classified the 33 original sampling locations into 17 homogeneous groups, which could provide a starting point for a later recalibration of the lake's monitoring network.     

Gaussian mixture discriminant analysis and sub-pixel land cover characterization in remote sensing

Mixture analysis is a necessary component for capturing sub-pixel heterogeneity in the characterization of land cover from remotely sensed images. Mixture analysis approaches in remote sensing vary from conventional linear mixture models to nonlinear neural network mixture models. Linear mixture models are fairly simple and generally result in poor mixture analysis accuracy. Neural network models can achieve much higher accuracy, but typically lack interpretability. In this paper we present a mixture discriminant analysis (MDA) model for inferring land cover fractions within forest stands from Landsat Thematic Mapper images. Specifically, individual class distributions are modeled as mixtures of subclasses of Gaussian distributions, and land cover fractions are estimated using the corresponding posterior probabilities. Compared to a benchmark study on accuracy of mixture models with Plumas National Forest data, this MDA model easily outperforms traditional linear mixture models and parallels the performance of the ARTMAP neural network mixture model. In other words, the MDA model is observed to successfully combine the performance characteristics of more complex neural network models (due to the nonlinear nature of its classification rules), with the ease of interpretation associated with linear mixture models (due to its relatively simple structure). MDA models therefore offer an attractive alternative for addressing the mixture modeling problem in remote sensing.     

Model-based approach for high-dimensional non-Gaussian visual data clustering and feature weighting

Clustering is the task of classifying patterns or observations into clusters or groups. Generally, clustering in high-dimensional feature spaces has a lot of complications such as: the unidentified or unknown data shape which is typically non-Gaussian and follows different distributions; the unknown number of clusters in the case of unsupervised learning; and the existence of noisy, redundant, or uninformative features which normally compromise modeling capabilities and speed. Therefore, high-dimensional data clustering has been a subject of extensive research in data mining, pattern recognition, image processing, computer vision, and other areas for several decades. However, most of existing researches tackle one or two problems at a time which is unrealistic because all problems are connected and should be tackled simultaneously. Thus, in this paper, we propose two novel inference frameworks for unsupervised non-Gaussian feature selection, in the context of finite asymmetric generalized Gaussian (AGG) mixture-based clustering. The choice of the AGG distribution is mainly due to its ability not only to approximate a large class of statistical distributions (e.g. impulsive, Laplacian, Gaussian and uniform distributions) but also to include the asymmetry. In addition, the two frameworks simultaneously perform model parameters estimation as well as model complexity (i.e., both model and feature selection) determination in the same step. This was done by incorporating a minimum message length (MML) penalty in the model learning step and by fading out the redundant densities in the mixture using the rival penalized EM (RPEM) algorithm, for first and second frameworks, respectively. Furthermore, for both algorithms, we tackle the problem of noisy and uninformative features by determining a set of relevant features for each data cluster. The efficiencies of the proposed algorithms are validated by applying them to real challenging problems namely action and facial expression recognition.     

纹理分析中的图模型

纹理作为一种重要的视觉特征,广泛应用于图像分析。高斯图模型（GGM）可以很好地描述有交互作用的高维数据,因此可用来建立图像纹理模型。根据纹理特征的局部马尔可夫性和高斯变量的条件回归之间的关系,将复杂的模型选择转变为较简单的变量选择,应用惩罚正则化技巧同步选择邻域和估计参数。提取基于图模型的纹理特征分析纹理,实验显示了很好的效果。因此,利用高斯图模型来构建纹理模型有很好的发展前景。     

一种新的基于高斯混合模型的线性判别分析

Fisher准则函数的前提条件就是假设每类样本数据满足单峰高斯分布,即各类样本在模式空间的分布近似椭球状,但是当训练样本数据较多且呈多峰分布时,传统的Fisher准则函数并不能准确反映样本数据的分布,显然基于Fisher准则函数的线性判别分析得到的最优判别矢量集也不是最优的。针对这种情况,通过引入高斯混合模型的概念,提出了一种新的基于高斯混合模型的线性判别分析方法,同时也给出了在该模型下的最优判别矢量集的直接求解方法,并通过实验证明了该算法的有效性。     

采用模型和得分非监督自适应的说话人识别

在说话人识别的研究中, 使用以前的测试语句信息对模型参数或者测试得分进行动态更新, 使模型可以更精确地反映测试语句和说话人模型之间的关系, 这种更新策略称为非监督模式, 这方面的研究对实际的说话人识别系统具有非常重要的意义. 本文除了采用非监督的说话人模型自适应更新方法之外, 还提出了非监督的得分域自适应算法: 首先采用双高斯函数对得分建立一个先验的得分模型, 利用最大后验概率准则对得分规整的模型进行调整. 在测试过程中, 采用得分域和模型域的非监督算法可以互相补充, 提高识别率, 在NIST SRE 2006年1训练语段-1测试语段数据库上, 使用模型域和得分域非监督自适应的系统能够取得等错误率4.3%和检测代价函数0.021的结果.     

Gaussian fields for semi-supervised regression and correspondence learning

Gaussian fields (GF) have recently received considerable attention for dimension reduction and semi-supervised classification. In this paper we show how the GF framework can be used for semi-supervised regression on high-dimensional data. We propose an active learning strategy based on entropy minimization and a maximum likelihood model selection method. Furthermore, we show how a recent generalization of the LLE algorithm for correspondence learning can be cast into the GF framework, which obviates the need to choose a representation dimensionality.     

Network intrusion detection: Evaluating cluster, discriminant, and logit analysis

This paper evaluates the statistical methodologies of cluster analysis, discriminant analysis, and Logit analysis used in the examination of intrusion detection data. The research is based on a sample of 1200 random observations for 42 variables of the KDD-99 database, that contains ‘normal’ and ‘bad’ connections. The results indicate that Logit analysis is more effective than cluster or discriminant analysis in intrusion detection. Specifically, according to the Kappa statistic that makes full use of all the information contained in a confusion matrix, Logit analysis (K = 0.629) has been ranked first, with second discriminant analysis (K = 0.583), and third cluster analysis (K = 0.460).     

医学图像的混合模型成份数估计

混合模型成份数估计是医学图像聚类分析和密度估计的关键。针对基于信息准则的佑计方法存在过拟合问题,提出了一种新的基于高斯混合模型特征函数的估计方法。首先定义医学图像高斯混合模型的特征函数,然后构造了一个基于特征函数的混合模型成份佑计准则,最后设计了该准则的实现算法。新的估计方法通过选择合适的参数调控对数特征函数,让惩罚函数起到平衡作用。模拟数据和真实数据实验表明,此方法确定的混合模型的成份数K比其他经典的信息准则方法确定的更合理,避免了医学图像的过拟合问题。