DEA-Based Piecewise Linear Discriminant Analysis

Nonlinear classification models have better classification performance than the linear classifiers. However, for many nonlinear classification problems, piecewise-linear discriminant functions can approximate nonlinear discriminant functions. In this study, we combine the algorithm of data envelopment analysis (DEA) with classification information, and propose a novel DEA-based classifier to construct a piecewise-linear discriminant function, in this classifier, the nonnegative conditions of DEA model are loosed and class information is added; Finally, experiments are performed using a UCI data set to demonstrate the accuracy and efficiency of the proposed model.     

A fully complex-valued radial basis function classifier for real-valued classification problems

In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier.     

DF-LDA tree: a nonlinear multilevel classifier for pattern recognition

This article presents a new nonlinear classifier by arranging linear classifiers in a tree structure. The proposed classifier, called the direct fractional-step linear discriminant (DF-LDA) tree, adopts a tree structure containing a DF-LDA at each node. The structure of the tree classifier evolves as the training proceeds, so there is no need to decide any parameters as a priori. Due to the many DF-LDAs arranged in the tree structure, classification performance of the proposed classifier is improved over single-shot DF-LDA. The proposed DF-LDA tree is tested on various synthetic and real datasets. Experimental results show that the proposed classifier leads to very satisfactory results in terms of classification accuracy.     

Back-propagation network and its configuration for blood vessel detection in angiograms.

A neural-network classifier for detecting vascular structures in angiograms was developed. The classifier consisted of a multilayer feedforward network window in which the center pixel was classified using gray-scale information within the window. The network was trained by using the backpropagation algorithm with the momentum term. Based on this image segmentation problem, the effect of changing network configuration on the classification performance was also characterized. Factors including topology, rate parameters, training sample set, and initial weights were systematically analyzed. The training set consisted of 75 selected points from a 256x256 digitized cineangiogram. While different network topologies showed no significant effect on performance, both the learning process and the classification performance were sensitive to the rate parameters. In a comparative study, the network demonstrated its superiority in classification performance. It was also shown that the trained neural-network classifier was equivalent to a generalized matched filter with a nonlinear decision tree.     

Combined SVM-Based Feature Selection and Classification

Feature selection is an important combinatorial optimisation problem in the context of supervised pattern classification. This paper presents four novel continuous feature selection approaches directly minimising the classifier performance. In particular, we include linear and nonlinear Support Vector Machine classifiers. The key ideas of our approaches are additional regularisation and embedded nonlinear feature selection. To solve our optimisation problems, we apply difference of convex functions programming which is a general framework for non-convex continuous optimisation. Experiments with artificial data and with various real-world problems including organ classification in computed tomography scans demonstrate that our methods accomplish the desired feature selection and classification performance simultaneously. Editor: Dale Schuurmans     

一种自适应的多类Boosting分类算法

许多实际问题涉及到多分类技术,该技术能有效地缩小用户与计算机之间的理解差异。在传统的多类Boosting方法中,多类损耗函数未必具有猜测背离性,并且多类弱学习器的结合被限制为线性的加权和。为了获得高精度的最终分类器,多类损耗函数应具有多类边缘极大化、贝叶斯一致性与猜测背离性。此外,弱学习器的缺点可能会限制线性分类器的性能,但它们的非线性结合可以提供较强的判别力。根据这两个观点,设计了一个自适应的多类Boosting分类器,即SOHPBoost算法。在每次迭代中,SOHPBoost算法能够利用向量加法或Hadamard乘积来集成最优的多类弱学习器。这个自适应的过程可以产生多类弱学习的Hadamard乘积向量和,进而挖掘出数据集的隐藏结构。实验结果表明,SOHPBoost算法可以产生较好的多分类性能。     

活体肝细胞磷31磁共振波谱数据分类

基于活体肝细胞的31P磁共振波谱图（31Phosphorus Magnetic Resonance Spectroscopy,31P-MRS）对肝细胞数据进行诊断,分为3种类型：肝癌、肝硬化和正常肝。分别运用线性分类器和二次分类器对数据分类,并在分类前进行了特征抽取。线性分类器和二次分类器在“留一法”中对上述3种类型的分类准确率分别约为81.37%、77.75%、92.30%和95.27%、99.89%、99.70%。实验证明二次分类器相对于线性分类器,明显地提高了分类准确率。     

模糊支持张量机图像分类算法及其应用

针对在小样本图像分类应用中,以向量空间作为输入的传统分类算法的不足,提出以张量理论为基础,结合模糊支持向量机思想的基于张量图像样本的模糊支持张量机分类器,利用张量表示图像样本,求解最优张量面。通过手写体数字图像样本实验仿真,验证该算法的性能,随后将其应用到羽绒菱节图像识别中进行对比,该算法较传统算法平均高出6.3%以上的识别率。实验证明该算法更适合应用于图像样本分类识别。     

Estimating impervious surfaces from medium spatial resolution imagery: a comparison between fuzzy classification and LSMA

Impervious surfaces are important environmental indicators and are related to many environmental issues, such as water quality, stream health and the urban heat island effect. Therefore, detailed impervious surface information is crucial for urban planning and environment management. To extract impervious surfaces from remote sensing imagery, many algorithms and techniques have been developed. However, there are still debates over the strengths and limitations of linear versus nonlinear algorithms in handling mixed pixels in the urban landscapes. In the meantime, although many previous studies have compared various techniques, few comparisons were made between linear and nonlinear techniques. The objective of this study is to compare the performance between nonlinear and linear methods for impervious surface extraction from medium spatial resolution imagery. A linear spectral mixture analysis (LSMA) and a fuzzy classifier were applied to three Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) images acquired on 5 April 2004, 16 June 2001 and 3 October 2000, which covered Marion County, Indiana, United States. An aerial photo of Marion County with a spatial resolution of 0.14 m was used for validation of estimation results. Six impervious surface maps were yielded, and an accuracy assessment was performed. The root mean square error (RMSE), the mean average error (MAE), and the coefficient of determination (R ²) were calculated to indicate the accuracy of impervious surface maps. The results show that the fuzzy classification outperformed LSMA in impervious surface estimation in all seasons. For the June image, LSMA yielded a result with an RMSE of 13.2%, while the fuzzy classifier yielded an RMSE of 12.4%. For the April image, LSMA yielded an accuracy of 21.1% and the fuzzy classifier yielded 17.0%. For the October image, LSMA yielded a result with an RMSE of 19.8%, but the fuzzy classifier yielded an RMSE of 17.5%. Moreover, a subset image of the commercial, high-density and low-density residential areas was selected in order to compare the effectiveness of the developed algorithms for estimating impervious surfaces of different land use types. The result shows that the fuzzy classification was more effective than LSMA in both high-density and low-density residential areas. These areas prevailed with mixed pixels in the medium resolution imagery, such as ASTER. The results from the tested commercial area had a very high RMSE value due to the prevalence of shade in the area. It is suggested that the fuzzy classifier based on the nonlinear assumption can handle mixed pixels more effectively than LSMA.     

A theoretical study of linear and nonlinear equalization innonlinear magnetic storage channels

We present methods to systematically design a feedforward neural-network detector from the knowledge of the channel characteristics. Its performance is compared with the conventional linear equalizer in a magnetic recording channel suffering from signal-dependent noise and nonlinear intersymbol interference. The superiority of the nonlinear schemes are clearly observed in all cases studied, especially in the presence of severe nonlinearity and noise. We also show that the decision boundaries formed by a theoretically derived neural-network classifier are geometrically close to those of a neural network trained by the backpropagation algorithm. The approach in this work is suitable for quantifying the gain in using a neural-network method as opposed to linear methods in the classification of noisy patterns.     

Artificial immune‐based supervised classifier for land‐cover classification

This paper explores the potential of an artificial immune‐based supervised classification algorithm for land‐cover classification. This classifier is inspired by the human immune system and possesses properties similar to nonlinear classification, self/non‐self identification, and negative selection. Landsat ETM+ data of an area lying in Eastern England near the town of Littleport are used to study the performance of the artificial immune‐based classifier. A univariate decision tree and maximum likelihood classifier were used to compare its performance in terms of classification accuracy and computational cost. Results suggest that the artificial immune‐based classifier works well in comparison with the maximum likelihood and the decision‐tree classifiers in terms of classification accuracy. The computational cost using artificial immune based classifier is more than the decision tree but less than the maximum likelihood classifier. Another data set from an area in Spain is also used to compare the performance of immune based supervised classifier with maximum likelihood and decision‐tree classification algorithms. Results suggest an improved performance with the immune‐based classifier in terms of classification accuracy with this data set, too. The design of an artificial immune‐based supervised classifier requires several user‐defined parameters to be set, so this work is extended to study the effect of varying the values of six parameters on classification accuracy. Finally, a comparison with a backpropagation neural network suggests that the neural network classifier provides higher classification accuracies with both data sets, but the results are not statistically significant.     

Fuzzy discriminant analysis with kernel methods

A novel fuzzy nonlinear classifier, called kernel fuzzy discriminant analysis (KFDA), is proposed to deal with linear non-separable problem. With kernel methods KFDA can perform efficient classification in kernel feature space. Through some nonlinear mapping the input data can be mapped implicitly into a high-dimensional kernel feature space where nonlinear pattern now appears linear. Different from fuzzy discriminant analysis (FDA) which is based on Euclidean distance, KFDA uses kernel-induced distance. Theoretical analysis and experimental results show that the proposed classifier compares favorably with FDA.     

Breast cancer diagnosis using genetic programming generated feature

This paper proposes a novel method for breast cancer diagnosis using the feature generated by genetic programming (GP). We developed a new feature extraction measure (modified Fisher linear discriminant analysis (MFLDA)) to overcome the limitation of Fisher criterion. GP as an evolutionary mechanism provides a training structure to generate features. A modified Fisher criterion is developed to help GP optimize features that allow pattern vectors belonging to different categories to distribute compactly and disjoint regions. First, the MFLDA is experimentally compared with some classical feature extraction methods (principal component analysis, Fisher linear discriminant analysis, alternative Fisher linear discriminant analysis). Second, the feature generated by GP based on the modified Fisher criterion is compared with the features generated by GP using Fisher criterion and an alternative Fisher criterion in terms of the classification performance. The classification is carried out by a simple classifier (minimum distance classifier). Finally, the same feature generated by GP is compared with a original feature set as the inputs to multi-layer perceptrons and support vector machine. Results demonstrate the capability of this method to transform information from high-dimensional feature space into one-dimensional space and automatically discover the relationship among data, to improve classification accuracy.     

基于记忆细胞剪切和非线性资源分配的人工免疫识别系统

为减少人工免疫识别系统(AIRS)的记忆细胞数量并提高AIRS的分类准确率,提出一种基于记忆细胞剪切和非线性资源分配的人工免疫识别系统(PNAIRS).PNAIRS采用样本属性离散化来压缩训练空间,利用记忆细胞剪切来淘汰低适应度细胞,并使用非线性资源分配来优化分类器.PNAIRS对6个UCI数据集进行分类测试,测试结果与其它分类算法结果对比,显示PNAIRS具有较小规模的记忆细胞群体和较高的分类准确率,而且算法运行速度快.这表明PNAIRS算法是一个性能良好的分类算法,具有潜在的应用价值.     

Web cache optimization with nonlinear model using object features

In this paper, Web cache optimization by utilizing syntactic features extracted from cache objects is studied. A nonlinear model is used to predict the value of each cache object by using features from the HTTP responses of the server, the access log of the cache, and from the HTML structure of the object. In a case study, linear and nonlinear models are used to classify about 50,000 HTML documents according to their popularity. The nonlinear model yields classification percentages of 64 and 74 for the documents to be stored or to be removed from the cache, respectively. A synthetic workload is then used to study the performance gain from the classifier in a conventional Least Recently Used cache model. The results suggest that the proposed approach can improve the performance of the cache substantially.     

Theoretical bounds of majority voting performance for a binary classification problem

A number of earlier studies that have attempted a theoretical analysis of majority voting assume independence of the classifiers. We formulate the majority voting problem as an optimization problem with linear constraints. No assumptions on the independence of classifiers are made. For a binary classification problem, given the accuracies of the classifiers in the team, the theoretical upper and lower bounds for performance obtained by combining them through majority voting are shown to be solutions of the corresponding optimization problem. The objective function of the optimization problem is nonlinear in the case of an even number of classifiers when rejection is allowed, for the other cases the objective function is linear and hence the problem is a linear program (LP). Using the framework we provide some insights and investigate the relationship between two candidate classifier diversity measures and majority voting performance.     

Skin segmentation using color pixel classification: analysis and comparison

This work presents a study of three important issues of the color pixel classification approach to skin segmentation: color representation, color quantization, and classification algorithm. Our analysis of several representative color spaces using the Bayesian classifier with the histogram technique shows that skin segmentation based on color pixel classification is largely unaffected by the choice of the color space. However, segmentation performance degrades when only chrominance channels are used in classification. Furthermore, we find that color quantization can be as low as 64 bins per channel, although higher histogram sizes give better segmentation performance. The Bayesian classifier with the histogram technique and the multilayer perceptron classifier are found to perform better compared to other tested classifiers, including three piecewise linear classifiers, three unimodal Gaussian classifiers, and a Gaussian mixture classifier.     

多层组合分类器研究

为了提高监督分类的精度，本文从组合分类器的结构出发，提出一种横向多层组合模型，并对这种模型的运行方式与组合特性进行分析。该模型每层含有一个分类器，每个分类器的输入和输出一起作为其后面一层的输入。我们将简单贝叶斯法与BP神经网络组合成两层分类器。实验结果表明，这种组合方式有效地提高了单个方法的分类精度。     

Land use classification with textural analysis and the aggregation technique using multi-temporal JERS-1 L-band SAR images

This paper describes a method of improving classification accuracy when using Synthetic Aperture Radar (SAR) images. The classifier used is a maximum likelihood classifier. Texture and textural feature images were made and used for classification. The accuracy of various classification methods was compared. As a result, it was found that the best classification was produced by the aggregation of the classified image when using texture images as additional inputs to the classifier. It is also shown that textural analysis and the aggregation technique are useful in the classification of SAR images.