Inhibition in multiclass classification

The role of inhibition is investigated in a multiclass support vector machine formalism inspired by the brain structure of insects. The so-called mushroom bodies have a set of output neurons, or classification functions, that compete with each other to encode a particular input. Strongly active output neurons depress or inhibit the remaining outputs without knowing which is correct or incorrect. Accordingly, we propose to use a classification function that embodies unselective inhibition and train it in the large margin classifier framework. Inhibition leads to more robust classifiers in the sense that they perform better on larger areas of appropriate hyperparameters when assessed with leave-one-out strategies. We also show that the classifier with inhibition is a tight bound to probabilistic exponential models and is Bayes consistent for 3-class problems. These properties make this approach useful for data sets with a limited number of labeled examples. For larger data sets, there is no significant comparative advantage to other multiclass SVM approaches.     

A local and global classification machine with collaborative mechanism

As an advanced local and global learning machine, the existing maxi–min margin machine (M⁴) still has its heavy time-consuming weakness. Inspired from the fact that covariance matrix of a dataset can characterize its data orientation and compactness globally, a novel large margin classifier called the local and global classification machine with collaborative mechanism (C²M) is constructed to circumvent this weakness in this paper. This classifier divides the whole global data into two independent models, and the final decision boundary is obtained by collaboratively combining two hyperplanes learned from two independent models. The proposed classifier C²M can be individually solved as a quadratic programming problem. The total training time complexity is \(O(2N^3)\) which is faster than \(O(N^4)\) of M⁴. C²M can be well defined with the clear geometrical interpretation and can also be justified from a theoretical perspective. As an additional contribution, it is shown that C²M can robustly leverage the global information from those datasets with overlapping class margins, while M⁴ does not use such global information. We also use the kernel trick and exploit C²M’s kernelized version. Experiments on toy and real-world datasets demonstrate that compared with M⁴, C²M is a more time-saving local and global learning machine.     

Combining predictions in pairwise classification: An optimal adaptive voting strategy and its relation to weighted voting

Weighted voting is the commonly used strategy for combining predictions in pairwise classification. Even though it shows good classification performance in practice, it is often criticized for lacking a sound theoretical justification. In this paper, we study the problem of combining predictions within a formal framework of label ranking and, under some model assumptions, derive a generalized voting strategy in which predictions are properly adapted according to the strengths of the corresponding base classifiers. We call this strategy adaptive voting and show that it is optimal in the sense of yielding a MAP prediction of the class label of a test instance. Moreover, we offer a theoretical justification for weighted voting by showing that it yields a good approximation of the optimal adaptive voting prediction. This result is further corroborated by empirical evidence from experiments with real and synthetic data sets showing that, even though adaptive voting is sometimes able to achieve consistent improvements, weighted voting is in general quite competitive, all the more in cases where the aforementioned model assumptions underlying adaptive voting are not met. In this sense, weighted voting appears to be a more robust aggregation strategy.     

结合全局和局部正则化的半监督二分类算法

针对在半监督分类问题中单独使用全局学习容易出现的在整个输入空间中较难获得一个优良的决策函数的问题,以及单独使用局部学习可在特定的局部区域内习得较好的决策函数的特点,提出了一种结合全局和局部正则化的半监督二分类算法。该算法综合全局正则项和局部正则项的优点,基于先验知识构建的全局正则项能平滑样本的类标号以避免局部正则项学习不充分的问题,通过基于局部邻域内样本信息构建的局部正则项使得每个样本的类标号具有理想的特性,从而构造出半监督二分类问题的目标函数。通过在标准二类数据集上的实验,结果表明所提出的算法其平均分类正确率和标准误差均优于基于拉普拉斯正则项方法、基于正则化拉普拉斯正则项方法和基于局部学习正则项方法。     

Extreme learning machine for regression and multiclass classification

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the "generalized" single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.     

基于全局与局部标签关系的多标签图像分类方法

针对多标签图像分类任务中存在的难以对标签间的相互作用建模和全局标签关系固化的问题,结合自注意力机制和知识蒸馏（KD）方法,提出了一种基于全局与局部标签关系的多标签图像分类方法（ML-GLLR）。首先,局部标签关系（LLR）模型使用卷积神经网络（CNN）、语义模块和双层自注意力（DLSA）模块对局部标签关系建模;然后,利用KD方法使LLR学习全局标签关系。在公开数据集MSCOCO2014和VOC2007上进行实验,LLR相较于基于图卷积神经网络多标签图像分类（ML-GCN）方法,在平均精度均值（mAP）上分别提高了0.8个百分点和0.6个百分点,ML-GLLR相较于LLR在mAP上分别进一步提高了0.2个百分点和1.3个百分点。实验结果表明,所提ML-GLLR不仅能对标签间的相互关系进行建模,也能避免全局标签关系固化的问题。     

Combining global and local matching of multiple features for precise item image retrieval

With the fast-growing of online shopping services, there are millions even billions of commercial item images available on the Internet. How to effectively leverage visual search method to find the items of users’ interests is an important yet challenging task. Besides global appearances (e.g., color, shape or pattern), users may often pay more attention to the local styles of certain products, thus an ideal visual item search engine should support detailed and precise search of similar images, which is beyond the capabilities of current search systems. In this paper, we propose a novel system named iSearch and global/local matching of local features are combined to do precise retrieval of item images in an interactive manner. We extract multiple local features including scale-invariant feature transform (SIFT), regional color moments and object contour fragments to sufficiently represent the visual appearances of items; while global and local matching of large-scale image dataset are allowed. To do this, an effective contour fragments encoding and indexing method is developed. Meanwhile, to improve the matching robustness of local features, we encode the spatial context with grid representations and a simple but effective verification approach using triangle relations constraints is proposed for spatial consistency filtering. The experimental evaluations show the promising results of our approach and system.     

Pedestrian gender classification using combined global and local parts-based convolutional neural networks

Pattern Analysis and Applications - The identification of a person’s gender plays an important role in various visual surveillance and monitoring applications which are growing more...     

Orthogonal incremental extreme learning machine for regression and multiclass classification

Single-hidden-layer feedforward networks with randomly generated additive or radial basis function hidden nodes have been theoretically proved that they can approximate any continuous function. Meanwhile, an incremental algorithm referred to as incremental extreme learning machine (I-ELM) was proposed which outperforms many popular learning algorithms. However, I-ELM may produce redundant nodes which increase the network architecture complexity and reduce the convergence rate of I-ELM. Moreover, the output weight vector obtained by I-ELM is not the least squares solution of equation Hβ = T. In order to settle these problems, this paper proposes an orthogonal incremental extreme learning machine (OI-ELM) and gives the rigorous proofs in theory. OI-ELM avoids redundant nodes and obtains the least squares solution of equation Hβ = T through incorporating the Gram–Schmidt orthogonalization method into I-ELM. Simulation results on nonlinear dynamic system identification and some benchmark real-world problems verify that OI-ELM learns much faster and obtains much more compact neural networks than ELM, I-ELM, convex I-ELM and enhanced I-ELM while keeping competitive performance.     

结合全局和局部约束的sLDA铁路扣件分类模型

针对监督潜在狄利克雷分布（sLDA）模型中测试图像缺乏标注,导致测试主题分布忽略目标结构的问题,提出一种结合全局和局部约束的sLDA（glc-sLDA）扣件图像分类模型。首先,人工标注训练图像,并在sLDA模型中学习得到含有结构信息的训练主题分布;然后,计算测试主题分布,将测试图像的类别概率作为全局约束,将测试图像子块与训练图像子块的主题相似程度作为局部约束;最后,以全局和局部约束的乘积为更新权值,对训练主题分布加权求和得到新的测试主题分布,并在Softmax分类器中得到测试图像的分类结果。实验结果表明,glc-sLDA模型能表达扣件结构信息,与sLDA相比,各类别的扣件图像区分性增强,分类误检率减小了55%。     

基于核空间相对密度的SVDD多类分类算法*

针对现有基于支持向量数据描述(SVDD)的多类分类算法未能充分利用重叠区域样本分布信息等问题,提出了一种基于核空间相对密度的SVDD多类分类算法DM-SVDD。该算法首先由SVDD确定包围每类数据的最小超球,然后计算位于最小超球重叠区域中每个样本在其同类样本间的相对密度,最后以各类样本相对密度的均值为标准,对重叠区域内的待测样本进行分类。实验结果表明,算法DM-SVDD是可行有效的。     

Graph-based PU learning for binary and multiclass classification without class prior

Knowledge and Information Systems - How can we classify graph-structured data only with positive labels? Graph-based positive-unlabeled (PU) learning is to train a binary classifier given only the...     

A transitivity analysis of bipartite rankings in pairwise multi-class classification

Many multi-class classification algorithms in statistics and machine learning typically combine several binary classifiers in order to construct an overall classifier. In the popular pairwise ensemble, one classifier is built for each pair of classes, resulting in pairwise bipartite rankings. In contrast, ordinal regression algorithms consider a single ranking function for several ordered classes. It is known in the literature that pairwise ensembles can be useful for ordinal regression. However, can single ranking models make a contribution to multi-class classification? The answer to this question should be affirmative, as supported by theoretical results presented in this article. We conduct a formal analysis of the consistency of pairwise bipartite rankings by uncovering the conditions under which they can be equivalently expressed in terms of a single ranking. Similar to the utility representability of pairwise preference relations, it turns out that transitivity plays a crucial role in the characterization of the ranking representability of pairwise bipartite rankings. To this end, we introduce the new concepts of strict ranking representability, a restrictive condition that can be verified easily, and AUC ranking representability, a practically more useful condition that is more difficult to verify. However, the link between pairwise bipartite rankings and dice games allows us to formulate necessary transitivity conditions for AUC ranking representability. A sufficient condition on the other hand is obtained by introducing a new type of transitivity that can be verified by solving an integer quadratic program.     

Detecting breakdowns in local coherence in the writing of Chinese English learners

This paper introduces CTutor, an automated writing evaluation (AWE) tool for detecting breakdowns in local coherence and reports on a study that applies it to the writing of Chinese L2 English learners. The program is based on Centering theory (CT), a theory of local coherence and salience. The principles of CT are first introduced and then the design and function of CTutor are described. The effectiveness and reliability of the program was evaluated in a study that compared performance by CTutor and two human raters on the analysis of local incoherence and provision of revision on learner essays. Intermediate Chinese English as a foreign language learners (n = 52) were divided into two groups: one receiving CTutor feedback and the other receiving feedback from human raters. Learners in both groups completed three essays; each of which involved the submission of a first draft, revision with feedback on local coherence quality and re‐submission. Our results from the comparison between CTutor and human experts showed that this software tool is able to detect local coherence breakdowns with moderate accuracy (F₁‐measure is around 0.4). There was also little difference between participants' responses to CTutor feedback and human feedback in terms of revision behaviour, with both feedback modes resulting in similar revision pattern. Potential use of the program in instructional settings is discussed.     

调整聚类假设联合成对约束半监督分类方法

当不同类别的样本严重重叠在分类边界时,由于聚类假设不能很好地反映出数据的真实分布,基于聚类假设的半监督分类方法的性能,可能比与之对立的监督分类方法更差。针对上述不安全的半监督分类问题,提出了调整聚类假设联合成对约束半监督分类方法（ACA-JPC-S3VM）。一方面,它将单个未标记样本到数据分布边界的距离融入到模型的学习中,能够一定程度上缓解此类情况下算法性能的下降程度;另一方面,它将成对约束信息引入,弥补了模型对监督信息利用方面的不足。在UCI数据集上的实验结果表明,ACA-JPC-S3VM方法的性能绝不会低于支持向量机（SVM）,且在标记样本数量为10时的平均准确率较SVM高出5个百分点;在图像分类数据集上的实验结果表明,直推式支持向量机（TSVM）等半监督分类方法出现了不同程度的不安全学习情形（即性能相近或低于SVM）,而ACA-JPC-S3VM却能安全地学习。因此,ACA-JPC-S3VM具有更好的安全性与正确性。     

From global to local and viceversa: uses of associative rule learning for classification in imprecise environments

We propose two models for improving the performance of rule-based classification under unbalanced and highly imprecise domains. Both models are probabilistic frameworks aimed to boost the performance of basic rule-based classifiers. The first model implements a global-to-local scheme, where the response of a global rule-based classifier is refined by performing a probabilistic analysis of the coverage of its rules. In particular, the coverage of the individual rules is used to learn local probabilistic models, which ultimately refine the predictions from the corresponding rules of the global classifier. The second model implements a dual local-to-global strategy, in which single classification rules are combined within an exponential probabilistic model in order to boost the overall performance as a side effect of mutual influence. Several variants of the basic ideas are studied, and their performances are thoroughly evaluated and compared with state-of-the-art algorithms on standard benchmark datasets.     

Combining fractal hourglass network and skeleton joints pairwise affinity for multi-person pose estimation

Multimedia Tools and Applications - Human pose estimation, especially multi-person pose estimation, is vital for understanding human abnormal behavior. In this paper, we develop a fractal hourglass...     

Combining image compression and classification using vectorquantization

We describe a method of combining classification and compression into a single vector quantizer by incorporating a Bayes risk term into the distortion measure used in the quantizer design algorithm. Once trained, the quantizer can operate to minimize the Bayes risk weighted distortion measure if there is a model providing the required posterior probabilities, or it can operate in a suboptimal fashion by minimizing the squared error only. Comparisons are made with other vector quantizer based classifiers, including the independent design of quantization and minimum Bayes risk classification and Kohonen's LVQ. A variety of examples demonstrate that the proposed method can provide classification ability close to or superior to learning VQ while simultaneously providing superior compression performance     

基于二叉树的SVM多类分类的研究与改进

支持向量机(SVM)是一种两类分类算法,如何将SVM算法应用于多类分类问题,目前已衍生出多种方法.其中“二叉树”方法应用比较广泛,但分类支持向量机在树中中间节点位置的不同,直接关系到该方法的分类准确性.基于二叉树方法提出了“类间相异度”的策略,根据类间相异程度来决定多类的分类顺序.     

Binary tree of SVM: a new fast multiclass training and classification algorithm

We present a new architecture named Binary Tree of support vector machine (SVM), or BTS, in order to achieve high classification efficiency for multiclass problems. BTS and its enhanced version, c-BTS, decrease the number of binary classifiers to the greatest extent without increasing the complexity of the original problem. In the training phase, BTS has N-1 binary classifiers in the best situation (N is the number of classes), while it has log/sub 4/3/((N+3)/4) binary tests on average when making a decision. At the same time the upper bound of convergence complexity is determined. The experiments in this paper indicate that maintaining comparable accuracy, BTS is much faster to be trained than other methods. Especially in classification, due to its Log complexity, it is much faster than directed acyclic graph SVM (DAGSVM) and ECOC in problems that have big class number.