Statistical approaches to combining binary classifiers for multi-class classification

One of the popular methods for multi-class classification is to combine binary classifiers. In this paper, we propose a new approach for combining binary classifiers. Our method trains a combining method of binary classifiers using statistical techniques such as penalized logistic regression, stacking, and a sparsity promoting penalty. Our approach has several advantages. Firstly, our method outperforms existing methods even if the base classifiers are well-tuned. Secondly, an estimate of conditional probability for each class can be naturally obtained. Furthermore, we propose selecting relevant binary classifiers by adding the group lasso type penalty in training the combining method.     

Meshless method to solve nonstationary heat conduction problems using atomic radial basis functions

The authors consider a meshless method to solve 3D nonstationary boundary-value heat conduction problems. It is implemented through an iterative scheme based on a combination of the double substitution method and the method of fundamental solutions with the use of atomic radial basis functions. The approaches to the visualization of the desired solution are considered.     

基于多目标优化的SVM多类分类方法

为了利用ROC曲线下的面积(AUC),更好地评价多类SVM学习效果,提出了MOSMAUC(multi-objectiveoptimizesmulti-classSVMbasedonAUC)算法.该算法采用AUC作为评价标准,利用多目标优化算法作为SVM参数的优化方法,避免优化对象的AUC值过低问题,因为在多类分类学习中任何一个两类分类的AUC值太低,都会影响整体学习的效果.实验结果表明,提出的优化方法改进了算法的学习能力,取得了较好的学习效果.     

An application of Non-Parametric Predictive Inference on multi-class classification high-level-noise problems

This paper presents an application of the Non-parametric Predictive Inference model for multinomial data (NPIM) on multiclass classification noise tasks, i.e. classification tasks where the variable under study has 3 or more possible states or values; and the data sets have incorrect class labels in their training and/or test data sets. In an experimental study, we show that the combination or fusion of the information obtained from decision trees built using the NPIM in a Bagging scheme, can improve the process of classification in multi-class classification noise problems. Via a set of statistical tests, we compared this approach with other successful methods used in similar scheme, on a wide set of data sets. It must be remarked that the new approach has a notably performance, compared with the rest of models, when the level of noise is increased.     

A genetic method for designing TSK models based on objective weighting: application to classification problems.

This paper proposes a genetic-based algorithm for generating simple and well-defined Takagi-Sugeno-Kang (TSK) models. The method handles several attributes simultaneously, such as the input partition, feature selection and estimation of the consequent parameters. The model building process comprises three stages. In stage one, structure learning is formulated as an objective weighting optimization problem. Apart from the mean square error (MSE) and the number of rules, three additional criteria are introduced in the fitness function for measuring the quality of the partitions. Optimization of these measures leads to models with representative rules, small overlapping and efficient data cover. To obtain models with good local interpretation, the consequent parameters are determined using a local MSE function while the overall model is evaluated on the basis of a global MSE function. The initial model is simplified at stage two using a rule base simplification routine. Similar fuzzy sets are merged and the “don’t care” premises are recognized. Finally, the simplified models are fine-tuned at stage three to improve the model performance. The suggested method is used to generate TSK models with crisp and polynomial consequents for two benchmark classification problems, the iris and the wine data. Simulation results reveal the effectiveness of our method. The resulting models exhibit simple structure, interpretability and superior recognition rates compared to other methods of the literature.     

Graphical method to solve combinatorial optimization problems

Proposed was a graphical method to solve decomposable problems of combinatorial optimization with the of use the Bellman optimality principle. In distinction to the dynamic programming algorithms based on the same principle, the graphical algorithm considers all possible system states by groups and not separately. This becomes possible if one takes into account the analytical form of the objective function, that is, handles the function “graph” and transforms it analytically at each stage. The graphical method enables one to reduce running time of solution of some problems and construct efficient approximation schemes. The results of numerical experiments corroborate efficiency of the of the graphical method.     

Use of probabilistic neural networks to solve classification problems in surgery

The paper considers the classification of peritonitis-stricken patients with regard to the outcome of the operation and the probable result of the treatment. A probabilistic neural network is offered as a classifier.     

Facing the classification of binary problems with a hybrid system based on quantum-inspired binary gravitational search algorithm and K-NN method

Since given classification data often contains redundant, useless or misleading features, feature selection is an important pre-processing step for solving classification problems. This problem is often solved by applying evolutionary algorithms to decrease the dimensional number of features involved. Removing irrelevant features in the feature space and identifying relevant features correctly is the primary objective, which can increase classification accuracy. In this paper, a novel QBGSA–K-NN hybrid system which hybridizes the quantum-inspired binary gravitational search algorithm (QBGSA) with the K-nearest neighbor (K-NN) method with leave-one-out cross-validation (LOOCV) is proposed. The main aim of this system is to improve classification accuracy with an appropriate feature subset in binary problems. We evaluate the proposed hybrid system on several UCI machine learning benchmark examples. The experimental results show that the proposed method is able to select the discriminating input features correctly and achieve high classification accuracy which is comparable to or better than well-known similar classifier systems.     

多分类光滑支持向量机的人脸识别方法

提出一种新的三阶分段光滑函数,构造三阶光滑支持向量机模型( TPSSVM)。理论证明新三阶分段光滑函数对正号函数的逼近程度更高。在处理多类问题时,提出一种基于编码方式的一对多光滑支持向量机分类方法。对于人脸识别问题,通过主成分分析( PCA)进行特征提取,并利用多分类光滑支持向量机对人脸特征图像进行训练和测试。应用于ORL人脸库和FERET人脸库的测试结果表明,多分类光滑支持向量机比传统的识别方法有更高的识别率。     

An ensemble approach of dual base learners for multi-class classification problems

In this work, we formalise and evaluate an ensemble of classifiers that is designed for the resolution of multi-class problems. To achieve a good accuracy rate, the base learners are built with pairwise coupled binary and multi-class classifiers. Moreover, to reduce the computational cost of the ensemble and to improve its performance, these classifiers are trained using a specific attribute subset. This proposal offers the opportunity to capture the advantages provided by binary decomposition methods, by attribute partitioning methods, and by cooperative characteristics associated with a combination of redundant base learners. To analyse the quality of this architecture, its performance has been tested on different domains, and the results have been compared to other well-known classification methods. This experimental evaluation indicates that our model is, in most cases, as accurate as these methods, but it is much more efficient.     

Feature extraction based on ICA for binary classification problems

In manipulating data such as in supervised learning, we often extract new features from the original features for the purpose of reducing the dimensions of feature space and achieving better performance. In this paper, we show how standard algorithms for independent component analysis (ICA) can be appended with binary class labels to produce a number of features that do not carry information about the class labels-these features will be discarded-and a number of features that do. We also provide a local stability analysis of the proposed algorithm. The advantage is that general ICA algorithms become available to a task of feature extraction for classification problems by maximizing the joint mutual information between class labels and new features, although only for two-class problems. Using the new features, we can greatly reduce the dimension of feature space without degrading the performance of classifying systems.     

Collocation method to solve inequality-constrained optimal control problems of arbitrary order

In this paper, the generalized fractional order of the Chebyshev functions (GFCFs) based on the classical Chebyshev polynomials of the first kind is used to obtain the solution of optimal control problems governed by inequality constraints. For this purpose positive slack functions are added to inequality conditions and then the operational matrix for the fractional derivative in the Caputo sense, reduces the problems to those of solving a system of algebraic equations. It is shown that the solutions converge as the number of approximating terms increases, and the solutions approach to classical solutions as the order of the fractional derivatives approach one. The applicability and validity of the method are shown by numerical results of some examples, moreover a comparison with the existing results shows the preference of this method.

     

The method of angular lines to solve cylindrical electromagnetic boundary-value problems

A method for solving cylindrical electromagnetic boundary-value problems is presented. The method is semidiscrete. The wave equation is written in cylindrical coordinates and a differential-discretizing process is used in angular direction, and in the radial direction the analytic solutions are used. The new method is called the method of angular lines (MAL). To confirm the validity of MAL Dirichlet- and Neumann-type boundary-value problems of a two-dimensional scalar wave equation are investigated. The results show good agreement with previously published results. Advantages and disadvantages of the method are discussed. © 1995 John Wiley & Sons, Inc.     

不平衡分类问题研究综述

实际的分类问题往往都是不平衡分类问题,采用传统的分类方法,难以得到满意的分类效果。为此,十多年来,人们相继提出了各种解决方案。对国内外不平衡分类问题的研究做了比较详细地综述,讨论了数据不平衡性引发的问题,介绍了目前几种主要的解决方案。通过仿真实验,比较了具有代表性的重采样法、代价敏感学习、训练集划分以及分类器集成在3个实际的不平衡数据集上的分类性能,发现训练集划分和分类器集成方法能较好地处理不平衡数据集,给出了针对不平衡分类问题的分类器评测指标和将来的工作。     

Second-order improvement method to solve problems of optimal control of the logic-dynamic systems

Consideration was given to the problem of optimal control of the logic-dynamic (mixed discrete-continuous in time and controllable) systems. A second-order improvement algorithm was developed for it on the basis of the Krotov sufficient optimality conditions. Examples were given.     

防止OFDM天线放大器峰值削波的AMAPR帧加权方法

OFDM系统由于子载波数目庞大,具有较大的动态信号范围和非常高的峰均功率比(PAPR),往往造成天线放大器的非线性失真和峰值削波,从而增加系统的误码率。较为先进的算法是利用峰值因数PAR对输入信号进行加权,降低了峰均功率比PAPR,但该算法使得输入信号大幅衰减,信噪比迅速减小,误码率增加。基于上述问题,提出新的,利用AMAPR(信号峰值与天线放大器极大值比)进行帧加权的计算方法,当某一帧最大功率大于放大器的线性区间,再对该帧实现线性补偿方法。逐帧计算加权系数,尽最大可能提高输入信号的信噪比。通过仿真,验证了AMAPR帧加权算法能防止峰值削波,改进误码率性能,防止信号的大幅度衰减,实现了低成本天线放大器的线性补偿。     

Algorithm to solve linear multipoint problems of control by the method of cascade decomposition

Proposed was a method to solve control problems for the nonstationary heterogeneous linear systems under conditions on the state and control functions at different time points. It consists of a phased transition from of the initial problem to the formula for the determination of some part of the functions of control and a problem similar to the previous one for a system with a smaller number of equations. The suggested method is a modification of the previous method of cascade decomposition, which solves problems using the projectors on the subspace. In the present paper, decomposition was carried out by solving the linear algebraic equations and replacing linearly the desired vector functions. The procedure of differentiation of certain functions was used to construct the controlled process.     

A CMOS binary pattern classifier based on Parzen''s method

Biological circuitry in the brain that has been associated with the Parzen method of classification inspired an analog CMOS binary pattern classifier. The circuitry resides on three separate chips. The first chip computes the closeness of a test vector to each training vector stored on the chip where "vector closeness" is defined as the number of bits two vectors have in common above some thresholds. The second chip computes the closeness of the test vector to each possible category where "category closeness" is defined as the sum of the closenesses of the test vector to each training vector in a particular category. Category closenesses are coded by currents which feed into an "early bird" winner-take-all circuit on the third chip that selects the category closest to the test vector. Parzen classifiers offer superior classification accuracy than the common nearest neighbor Hamming networks. A high degree of parallelism allows for O(1) time complexity and the chips are tillable for increased training vector storage capacity. Proof-of-concept chips were fabricated through the MOSIS chip prototyping service and successfully tested.