基于线性判别回归的最近-最远子空间分类鲁棒人脸识别

针对人脸识别中小样本问题导致类依赖子空间不完善而严重影响识别性能的问题,提出一种基于线性判别回归的最近-最远子空间分类算法。首先,基于线性判别回归,利用最近子空间分类器度量测试图像与单一类之间的关系;然后,利用所提出的最远子空间分类器度量测试图像与训练图像之间的关系;最后,结合最近、最远子空间分类器,利用类依赖子空间的不同特性完成人脸的分类识别。在三个公开的人脸数据库ORL、AR及扩展Yale B上的实验验证了该算法的有效性。实验结果表明,相比其他几种分类算法,该算法取得了更好的识别效果。     

Mixed-Fisher特征云模型聚类在文本情感分类中的应用

海量网络信息的出现,使得提取文本信息情感观点成为研究的热点。针对文本情感分类中文本信息模糊及分类准确率低的问题,提出了一种基于Mixed-Fisher特征选择的文本云向量模型聚类算法。该算法首先分别计算文档中各个词性特征项的Fisher判别比,根据Fisher判别比越大特征向量判别性越强的Fisher准则,选择Fisher比值较大的前q个特征,并按照词性进行组合生成文档的Mixed-Fisher特征向量。然后在Mixed-Fisher特征向量集上构建文档的云向量模型,根据云向量模型间的差异度对模型进行聚类和合并。将该算法应用于文本情感观点的分类,选择核Fisher判别技术用于最终文本观点的判定。仿真实验结果表明,基于Mixed-Fisher特征的云向量聚类模型的分类准确率明显优于传统向量空间模型,从而验证了核Fisher判别技术的有效性。     

弹性核子空间聚类*

现有子空间聚类算法通常假设数据来自多个线性子空间,无法处理时间序列聚类中存在的非线性和时间轴弯曲问题.为了克服这些局限,通过引入核技巧和弹性距离,提出弹性核低秩表示子空间聚类和弹性核最小二乘回归子空间聚类,统称为弹性核子空间聚类,并从理论上证明弹性核最小二乘回归子空间算法的组效应和弹性核低秩表示子空间聚类算法的收敛性.在5个UCR时间序列数据集上的实验表明本文算法的有效性.     

基于Fisher准则改进线性判别回归分类的人脸识别

为了提高线性回归分类LRC(Linear Regression Classification)算法的鲁棒性,提出一种基于Fisher准则改进的线性判别回归分类算法。首先根据Fisher准则最大化类间重建误差与类内重建误差的比值,为LRC找到最优投影矩阵;然后利用最优投影矩阵将训练图像及测试图像投影到各个类的特征子空间;最后,计算出测试图像与各个训练图像之间的欧氏距离,并利用K-近邻分类器完成人脸的识别。在FERET和AR人脸数据库上的实验验证了本文算法的有效性。实验结果表明,相比其他回归分类算法,该算法取得了更好的识别效果。     

系数增强最小二乘回归子空间聚类法

针对最小二乘回归子空间聚类法在求解表示系数时忽略了样本相似度的不足,提出改进方法。基于样本相互重构的表示系数矩阵和样本相似度矩阵有很大的关联定义系数增强项,求解可以保持样本相似度的表示系数矩阵,提出系数增强最小二乘回归子空间聚类法。在8个标准数据集上的实验表明该方法可以提高最小二乘回归子空间聚类法的聚类性能。     

基于典型相关分析的多视图跨领域情感分类

跨领域的文本分类,是指利用有标记领域的知识去帮助另一个概率分布不同的,未标记领域的知识进行分类的问题。从多视图学习的视角提出一个新的跨领域文本分类的方法(MTV算法)。通过在核空间典型相关分析中引入与标记相关的信息,MTV算法可以得到一个判别性能更优的公共子空间。在多个情感类文本数据上的实验表明,MTV算法可以大大提升传统监督式学习算法面对领域迁移时的分类性能,并且在引入判别式的核空间典型相关分析后,进一步优化性能。     

结构化稀疏低秩子空间聚类

通过定义子空间结构化低秩正则项,将其与子空间结构化稀疏子空间聚类模型相结合,给出一个新的统一优化模型。新模型利用数据的类别属性和相似度互相引导,使得相似度具有判别性和一致性,类别属性具有一致性。相似度的判别性有利于将不同子空间的数据分为不同类,而一致性有利于将同一子空间的数据聚为一类。大量实验表明提出的方法优于一些典型的两步法和子空间结构化稀疏子空间聚类模型。     

改进型低秩线性回归新方法

对于遮挡、光照等影响因素,低秩线性回归模型具有很好的鲁棒性。LRRR(Low Rank Ridge Regression)以及DENLR(Discriminative Elastic-net Regularized Linear Regression)通过正则化系数矩阵在一定程度上减少了LRLR(Low Rank Linear Regression)产生的过拟合现象。但其没有考虑子空间数据的错误逼近,投影矩阵不能准确地将数据映射到目标空间。鉴于此,提出了一种运算更快、更具判别性的低秩线性回归分类新方法。首先,将0-1构成的矩阵作为线性回归的目标值;其次,利用核范数作为低秩约束的凸近似;然后,通过正则化各类别之间的距离矩阵和模型输出矩阵来降低过拟合,同时可以增强投影子空间的判别性;再次,利用增广拉格朗日乘子(Augmented Lagrangian Multiplier,ALM)优化目标函数;最后,在子空间中利用最近邻分类器进行分类。在AR、FERET人脸数据库、Stanford 40 Actions、Caltech-UCSD Bird以及Oxford 102 Flowers数据库上进行相关算法的对比实验,结果表明所提算法是有效的。     

改进的多层判别回归树算法及其在遥感图像分析中的应用

该文针对高维数据的快速聚类与回归问题及其在遥感图像分析中的应用等问题进行研究 .文中分析了改进的多层判别回归树的输入空间和和输出空间的联系 ,设计了改进的多层判别回归树的构造和检索算法 ,并且分析了它们的算法复杂度 ,还给出了系统实现和测试结果 ,最后在巡航导弹的地形匹配制导或弹道校正、遥感图像的城市绿化面积的估算以及纹理分析等应用领域进行了实践 .     

核聚类算法

该文提出了一种用于聚类分析的核聚类方法，通过利用Mercer核，作者把输入空间的样本映射到高维特征空间后，在特征空间中进行聚类，由于经过了核函数的映射，使原来没有显现的特征突出来，从而能够更好地聚类，该核聚类方法在性能上比以典的聚类算法有较大的改进，具有更快的收敛速度以及更为准确的聚类，仿真实验的结果证实了核聚类方法的可行性和有效性。     

Hierarchical discriminant regression

The main motivation of this paper is to propose a classification and regression method for challenging high-dimensional data. The proposed technique casts classification problems and regression problems into a unified regression problem. This unified view enables classification problems to use numeric information in the output space that is available for regression problems but are traditionally not readily available for classification problems. A doubly clustered subspace-based hierarchical discriminating regression (HDR) method is proposed. The major characteristics include: (1) Clustering is performed in both output space and input space at each internal node, termed "doubly clustered." Clustering in the output space provides virtual labels for computing clusters in the input space. (2) Discriminants in the input space are automatically derived from the clusters in the input space. (3) A hierarchical probability distribution model is applied to the resulting discriminating subspace at each internal node. This realizes a coarse-to-fine approximation of probability distribution of the input samples, in the hierarchical discriminating subspaces. (4) To relax the per class sample requirement of traditional discriminant analysis techniques, a sample-size dependent negative-log-likelihood (NLL) is introduced. This new technique is designed for automatically dealing with small-sample applications, large-sample applications, and unbalanced-sample applications. (5) The execution of the HDR method is fast, due to the empirical logarithmic time complexity of the HDR algorithm. Although the method is applicable to any data, we report the experimental results for three types of data: synthetic data for examining the near-optimal performance, large raw face-image databases, and traditional databases with manually selected features along with a comparison with some major existing methods.     

Online image classification using IHDR

This paper presents an incremental algorithm for image classification problems. Virtual labels are automatically formed by clustering in the output space. These virtual labels are used for the process of deriving discriminating features in the input space. This procedure is performed recursively in a coarse-to-fine fashion resulting in a tree, performing incremental hierarchical discriminating regression (IHDR). Embedded in the tree is a hierarchical probability distribution model used to prune unlikely cases. A sample size dependent negative-log-likelihood (NLL) metric is introduced to deal with large sample-size cases, small sample-size cases, and unbalanced sample-size cases, measured among different internal nodes of the IHDR algorithm. We report the experimental results of the proposed algorithm for an OCR classification problem and an image orientation classification problem. Received: November 20, 2001 / Accepted: May 10, 2002     

Multi-output regression on the output manifold

Multi-output regression aims at learning a mapping from an input feature space to a multivariate output space. Previous algorithms define the loss functions using a fixed global coordinate of the output space, which is equivalent to assuming that the output space is a whole Euclidean space with a dimension equal to the number of the outputs. So the underlying structure of the output space is completely ignored. In this paper, we consider the output space as a Riemannian submanifold to incorporate its geometric structure into the regression process. To this end, we propose a novel mechanism, called locally linear transformation (LLT), to define the loss functions on the output manifold. In this way, currently existing regression algorithms can be improved. In particular, we propose an algorithm under the support vector regression framework. Our experimental results on synthetic and real-life data are satisfactory.     

集成Logistic与SVM的二分类算法

对Logistic回归的输出结果通过概率分析划分为四个连续的区间,计算各个区间内训练样本的正确分类频率,由此将Logistic回归与支持向量机对样本的输出结果进行比较,构造了一种集成判别规则的二分类算法。实证分析表明提出的集成算法具有较好的分类效果。     

Local Dimensionality Reduction for Non-Parametric Regression

Locally-weighted regression is a computationally-efficient technique for non-linear regression. However, for high-dimensional data, this technique becomes numerically brittle and computationally too expensive if many local models need to be maintained simultaneously. Thus, local linear dimensionality reduction combined with locally-weighted regression seems to be a promising solution. In this context, we review linear dimensionality-reduction methods, compare their performance on non-parametric locally-linear regression, and discuss their ability to extend to incremental learning. The considered methods belong to the following three groups: (1) reducing dimensionality only on the input data, (2) modeling the joint input-output data distribution, and (3) optimizing the correlation between projection directions and output data. Group 1 contains principal component regression (PCR); group 2 contains principal component analysis (PCA) in joint input and output space, factor analysis, and probabilistic PCA; and group 3 contains reduced rank regression (RRR) and partial least squares (PLS) regression. Among the tested methods, only group 3 managed to achieve robust performance even for a non-optimal number of components (factors or projection directions). In contrast, group 1 and 2 failed for fewer components since these methods rely on the correct estimate of the true intrinsic dimensionality. In group 3, PLS is the only method for which a computationally-efficient incremental implementation exists. Thus, PLS appears to be ideally suited as a building block for a locally-weighted regressor in which projection directions are incrementally added on the fly.     

Technical Note: Naive Bayes for Regression

Despite its simplicity, the naive Bayes learning scheme performs well on most classification tasks, and is often significantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is categorical. It is therefore interesting to see how it performs in domains where the predicted value is numeric, because in this case, predictions are more sensitive to inaccurate probability estimates.This paper shows how to apply the naive Bayes methodology to numeric prediction (i.e., regression) tasks by modeling the probability distribution of the target value with kernel density estimators, and compares it to linear regression, locally weighted linear regression, and a method that produces model trees—decision trees with linear regression functions at the leaves. Although we exhibit an artificial dataset for which naive Bayes is the method of choice, on real-world datasets it is almost uniformly worse than locally weighted linear regression and model trees. The comparison with linear regression depends on the error measure: for one measure naive Bayes performs similarly, while for another it is worse. We also show that standard naive Bayes applied to regression problems by discretizing the target value performs similarly badly. We then present empirical evidence that isolates naive Bayes' independence assumption as the culprit for its poor performance in the regression setting. These results indicate that the simplistic statistical assumption that naive Bayes makes is indeed more restrictive for regression than for classification.     

An incremental adaptive neural network model for online noisy data regression and its application to compartment fire studies

This paper presents a probabilistic-entropy-based neural network (PENN) model for tackling online data regression problems. The network learns online with an incremental growth network structure and performs regression in a noisy environment. The training samples presented to the model are clustered into hyperellipsoidal Gaussian kernels in the joint space of the input and output domains by using the principles of Bayesian classification and minimization of entropy. The joint probability distribution is established by applying the Parzen density estimator to the kernels. The prediction is carried out by evaluating the expected conditional mean of the output space with the given input vector. The PENN model is demonstrated to be able to remove symmetrically distributed noise embedded in the training samples. The performance of the model was evaluated by three benchmarking problems with noisy data (i.e., Ozone, Friedman#1, and Santa Fe Series E). The results show that the PENN model is able to outperform, statistically, other artificial neural network models. The PENN model is also applied to solve a fire safety engineering problem. It has been adopted to predict the height of the thermal interface which is one of the indicators of fire safety level of the fire compartment. The data samples are collected from a real experiment and are noisy in nature. The results show the superior performance of the PENN model working in a noisy environment, and the results are found to be acceptable according to industrial requirements.     

Technology credit scoring model with fuzzy logistic regression

Technology credit scoring models have been used to screen loan applicant firms based on their technology. Typically a logistic regression model is employed to relate the probability of a loan default of the firms with several evaluation attributes associated with technology. However, these attributes are evaluated in linguistic expressions represented by fuzzy number. Besides, the possibility of loan default can be described in verbal terms as well. To handle these fuzzy input and output data, we proposed a fuzzy credit scoring model that can be applied to predict the default possibility of loan for a firm that is approved based on its technology. The method of fuzzy logistic regression as an appropriate prediction approach for credit scoring with fuzzy input and output was presented in this study. The performance of the model is improved compared to that of typical logistic regression. This study is expected to contribute to practical utilization of the technology credit scoring with linguistic evaluation attributes.     

基于径向基函数的多目标回归特征构建算法

多目标回归（MTR）是一种针对单个样本同时具有多个连续型输出的回归问题。现有的多目标回归算法都基于同一个特征空间学习回归模型,而忽略了各输出目标本身的特殊性质。针对这一问题,提出基于径向基函数的多目标回归特征构建算法。首先,将各目标的输出作为额外的特征对各输出目标进行聚类,根据聚类中心在原始特征空间构成了目标特定特征空间的基;然后,通过径向基函数将原始特征空间映射到目标特定特征空间,构造目标特定的特征,并基于这些目标特定特征构建各输出目标的基回归模型;最后,用基回归模型的输出组成隐藏空间,采用低秩学习算法在其中发掘和利用输出目标之间的关联。在18个多目标回归数据集上进行实验,并把所提算法与层叠单目标回归（SST）、回归器链集成（ERC）、多层、多目标回归（MMR）等经典的多目标回归算法进行对比,结果表明所提算法在14个数据集上都取得了最好的性能,并且在18个数据集上的平均性能排序居第一位。可见所提算法构建的目标特定特征能够提高各输出目标的预测准确性,并结合低秩学习得到输出目标间的关联性以从整体上提升多目标回归的预测性能。     

A non-intrusive stochastic Galerkin approach for modeling uncertainty propagation in deformation processes

Large deformation processes are inherently complex considering the non-linear phenomena that need to be accounted for. Stochastic analysis of these processes is a formidable task due to the numerous sources of uncertainty and the various random input parameters. As a result, uncertainty propagation using intrusive techniques requires tortuous analysis and overhaul of the internal structure of existing deterministic analysis codes. In this paper, we present an approach called non-intrusive stochastic Galerkin (NISG) method, which can be directly applied to presently available deterministic legacy software for modeling deformation processes with minimal effort for computing the complete probability distribution of the underlying stochastic processes. The method involves finite element discretization of the random support space and piecewise continuous interpolation of the probability distribution function over the support space with deterministic function evaluations at the element integration points. For the hyperelastic–viscoplastic large deformation problems considered here with varying levels of randomness in the input and boundary conditions, the NISG method provides highly accurate estimates of the statistical quantities of interest within a fraction of the time required using existing Monte Carlo methods.