On the effect of calibration in classifier combination

A general approach to classifier combination considers each model as a probabilistic classifier which outputs a class membership posterior probability. In this general scenario, it is not only the quality and diversity of the models which are relevant, but the level of calibration of their estimated probabilities as well. In this paper, we study the role of calibration before and after classifier combination, focusing on evaluation measures such as MSE and AUC, which better account for good probability estimation than other evaluation measures. We present a series of findings that allow us to recommend several layouts for the use of calibration in classifier combination. We also empirically analyse a new non-monotonic calibration method that obtains better results for classifier combination than other monotonic calibration methods.     

SC3: self-configuring classifier combination for obstructive sleep apnea

Neural Computing and Applications - Obstructive sleep apnea is considered to be one of the most prevalent sleep-related disorders that can affect the general population. However, the gold standard...     

Representing classifier confidence in the safety critical domain: an illustration from mortality prediction in trauma cases

This work proposes a novel approach to assessing confidence measures for software classification systems in demanding applications such as those in the safety critical domain. Our focus is the Bayesian framework for developing a model-averaged probabilistic classifier implemented using Markov chain Monte Carlo (MCMC) and where appropriate its reversible jump variant (RJ-MCMC). Within this context we suggest a new technique, building on the reject region idea, to identify areas in feature space that are associated with “unsure” classification predictions. We term such areas “uncertainty envelopes” and they are defined in terms of the full characteristics of the posterior predictive density in different regions of the feature space. We argue this is more informative than use of a traditional reject region which considers only point estimates of predictive probabilities. Results from the method we propose are illustrated on synthetic data and also usefully applied to real life safety critical systems involving medical trauma data.     

Multiple classifier decision combination strategies for character recognition: A review

Two research strands, each identifying an area of markedly increasing importance in the current development of pattern analysis technology, underlie the review covered by this paper, and are drawn together to offer both a task-oriented and a fundamentally generic perspective on the discipline of pattern recognition. The first of these is the concept of decision fusion for high-performance pattern recognition, where (often very diverse) classification technologies, each providing complementary sources of information about class membership, can be integrated to provide more accurate, robust and reliable classification decisions. The second is the rapid expansion in technology for the automated analysis of (especially) handwritten data for OCR applications including document and form processing, pen-based computing, forensic analysis, biometrics and security, and many other areas, especially those which seek to provide online or offline processing of data which is available in a human-oriented medium. Classifier combination/multiple expert processing has a long history, but the sheer volume and diversity of possible strategies now available suggest that it is timely to consider a structured review of the field. Handwritten character processing provides an ideal context for such a review, both allowing engagement with a problem area which lends itself ideally to the performance enhancements offered by multi-classifier configurations, but also allowing a clearer focus to what otherwise, because of the unlimited application horizons, would be a task of unmanageable proportions. Hence, this paper explicitly reviews the field of multiple classifier decision combination strategies for character recognition, from some of its early roots to the present day. In order to give structure and a sense of direction to the review, a new taxonomy for categorising approaches is defined and explored, and this both imposes a discipline on the presentation of the material available and helps to clarify the mechanisms by which multi-classifier configurations deliver performance enhancements. The review incorporates a discussion both of processing structures themselves and a range of important related topics which are essential to maximise an understanding of the potential of such structures. Most importantly, the paper illustrates explicitly how the principles underlying the application of multi-classifier approaches to character recognition can easily generalise to a wide variety of different task domains.Received: 3 February 2002, Accepted: 9 October 2002, Published online: 6 June 2003     

A support vector machine classifier with automatic confidence and its application to gender classification

In this paper, we propose a support vector machine with automatic confidence (SVMAC) for pattern classification. The main contributions of this work to learning machines are twofold. One is that we develop an algorithm for calculating the label confidence value of each training sample. Thus, the label confidence values of all of the training samples can be considered in training support vector machines. The other one is that we propose a method for incorporating the label confidence value of each training sample into learning and derive the corresponding quadratic programming problems. To demonstrate the effectiveness of the proposed SVMACs, a series of experiments are performed on three benchmarking pattern classification problems and a challenging gender classification problem. Experimental results show that the generalization performance of our SVMACs is superior to that of traditional SVMs.     

Reducing preference elicitation in group decision making

Groups may need assistance in reaching a joint decision. Elections can reveal the winning item, but this means the group members need to vote on, or at least consider all available items. Our challenge is to minimize the amount of preferences that need to be elicited and thus reduce the effort required from the group members. We present a model that offers a few innovations. First, rather than offering a single winner, we propose to offer the group the best top-k alternatives. This can be beneficial if a certain item suddenly becomes unavailable, or if the group wishes to choose manually from a few selected items. Secondly, rather than offering a definite winning item, we suggest to approximate the item or the top-k items that best suit the group, according to a predefined confidence level. We study the tradeoff between the accuracy of the proposed winner item and the amount of preference elicitation required. Lastly, we offer to consider different preference aggregation strategies. These strategies differ in their emphasis: towards the individual users (Least Misery Strategy) or towards the majority of the group (Majority Based Strategy). We evaluate our findings on data collected in a user study as well as on real world and simulated datasets and show that selecting the suitable aggregation strategy and relaxing the termination condition can reduce communication cost up to 90%. Furthermore, the commonly used Majority strategy does not always outperform the Least Misery strategy. Addressing these three challenges contributes to the minimization of preference elicitation in expert systems.     

Selected tree classifier combination based on both accuracy and error diversity

This paper proposes a method for combining multiple tree classifiers based on both classifier ensemble (bagging) and dynamic classifier selection schemes (DCS). The proposed method is composed of the following procedures: (1) building individual tree classifiers based on bootstrap samples; (2) calculating the distance between all possible two trees; (3) clustering the trees based on single linkage clustering; (4) selecting two clusters by local region in terms of accuracy and error diversity; and (5) voting the results of tree classifiers selected in the two clusters. Empirical evaluation using publicly available data sets confirms the superiority of our proposed approach over other classifier combining methods.     

基于组合特征和级联分类器的防震锤检测算法

提出一种高压输电线路上的防震锤检测识别算法,算法基于分块的Haar特征、基于区域的LBP特征以及HOG特征一起作为组合特征来检测防震锤。其主要分为5个步骤:预处理待检测图像;改进归一化互相关匹配算法并进行模板匹配,得到防震锤疑似区域样本集;提取防震锤疑似区域的组合特征;对防震锤疑似区域使用级联分类器进行多级分类;统计分类结果。实验结果表明,该算法具有较高的精确率、召回率和准确率。     

群决策中模糊偏好信息转化的若干性质研究

研究群体决策中偏好信息转化的一致性和权重变化问题.在一致性方面,得到了不同转化系数转化后的互反判断矩阵次序一致性、完全乘性一致性与互补判断矩阵相同,且一致性比例与转化系数的大小正相关的结论.在权重数值方面,证明了方案优先顺序与转化系数大小无关,但较大转化系数能放大方案之间的差别,而较小系数则缩小方案之间差别的性质.相关结论可为互补判断矩阵的权重求解以及一致性分析提供参考.     

Intuitionistic preference relations and their application in group decision making

Intuitionistic fuzzy set, characterized by a membership function and a non-membership function, was introduced by Atanassov [Intuitionistic fuzzy sets, Fuzzy Sets and Systems 20 (1986) 87-96]. In this paper, we define the concepts of intuitionistic preference relation, consistent intuitionistic preference relation, incomplete intuitionistic preference relation and acceptable intuitionistic preference relation, and study their various properties. We develop an approach to group decision making based on intuitionistic preference relations and an approach to group decision making based on incomplete intuitionistic preference relations respectively, in which the intuitionistic fuzzy arithmetic averaging operator and intuitionistic fuzzy weighted arithmetic averaging operator are used to aggregate intuitionistic preference information, and the score function and accuracy function are applied to the ranking and selection of alternatives. Finally, a practical example is provided to illustrate the developed approaches.     

一种新的李群分类器在手写体数字中的应用

李群是变换空间的一种基本表示理论。目前针对李群数据所设计的分类器较少,对多分类的效果也不是很好。以手写体数字的应用为背景,引入了支持向量机分类算法来处理李群数据。由于李群数据具有矩阵表现的形式,设计了一种矩阵高斯核函数,使得支持向量机能够处理矩阵数据。仿真结果表明,支持向量机方法在李群数据上具有很好的性能。     

Correlation‐based classifier combination in the field of pattern recognition

Classifier combination methods have proved to be an effective tool to increase the performance of classification techniques that can be used in any pattern recognition applications. Despite a significant number of publications describing successful classifier combination implementations, the theoretical basis is still not matured enough and achieved improvements are inconsistent. In this paper, we propose a novel statistical validation technique known as correlation‐based classifier combination technique for combining classifier in any pattern recognition problem. This validation has significant influence on the performance of combinations, and their utilization is necessary for complete theoretical understanding of combination algorithms. The analysis presented is statistical in nature but promises to lead to a class of algorithms for rank‐based decision combination. The potentials of the theoretical and practical issues in implementation are illustrated by applying it on 2 standard datasets in pattern recognition domain, namely, handwritten digit recognition and letter image recognition datasets taken from UCI Machine Learning Database Repository ( http://www.ics.uci.edu/_mlearn ). 1 An empirical evaluation using 8 well‐known distinct classifiers confirms the validity of our approach compared to some other combinations of multiple classifiers algorithms. Finally, we also suggest a methodology for determining the best mix of individual classifiers.     

使用群体兴趣偏好度的协同过滤推荐

推荐系统是电子商务系统中最重要的技术之一,用户相似性度量方法是影响推荐算法准确率高低的关键因素。针对用户评分数据极端稀疏情况下传统相似性度量方法的不足,提出了一种基于群体兴趣偏好度的协同过滤推荐算法,根据群体兴趣偏好度来预测用户对未评分项目的评分,在此基础上再采用传统的相似性度量方法计算目标用户的最近邻居。实验结果表明,该算法可以有效解决用户评分数据极端稀疏情况下传统相似性度量方法存在的问题,显著提高推荐系统的推荐质量。     

Developing complex group products: Idea combination in computer-mediated and face-to-face groups

This study examines how group experience, communication medium, and strategies for combining individual ideas influence the integrative complexity of group products. Each week for six weeks, members of 31 work groups wrote individual essays about their group tasks and experiences, and then collaborated on a group essay on the same topic. Results indicate that in the later weeks of the study, computer-mediated groups produce essays with higher integrative complexity than those of face-to-face groups. The integrative complexity of essays in later weeks is a joint function of the complexity of member ideas and the number of members who participate directly in writing the essay (scribes). The greater complexity of computer-mediated groups' essays in the later weeks of the study is partly accounted for by their use of more scribes and their inclusion of more unique member ideas.     

A morphologically optimal strategy for classifier combination: multiple expert fusion as a tomographic process

We specify an analogy in which the various classifier combination methodologies are interpreted as the implicit reconstruction, by tomographic means, of the composite probability density function spanning the entirety of the pattern space, the process of feature selection in this scenario amounting to an extremely bandwidth-limited Radon transformation of the training data. This metaphor, once elaborated, immediately suggests techniques for improving the process, ultimately defining, in reconstructive terms, an optimal performance criterion for such combinatorial approaches.     

The fusion process with heterogeneous preference structures in group decision making: A survey

In group decision making (GDM), decision makers who have different experiential, cultural and educational backgrounds will naturally provide their preference information by heterogeneous preference structures (e.g., utility values, preference orderings, numerical preference relations and multigranular linguistic preference relations). To date, many studies have discussed GDM problems with heterogeneous preference structures. To provide a clear perspective on the fusion process with heterogeneous preference structures in GDM, this paper presents a review of three types of fusion approaches: the indirect approach, the optimization-based approach and the direct approach. Moreover, with respect to insights gained from prior researches, several open problems are proposed for the future research.     

A loss function approach to group preference aggregation in the AHP

The Analytic Hierarchy Process is a useful method in aggregating group preference. However, judgments are frequently inconsistent, and, in reality, pairwise comparison matrices rarely satisfy the inconsistency criterion. In this situation, we suggest a new method, called a loss function approach, that uses inconsistency ratio as the group evaluation quality. For this method in detail, we introduce Taguchi's loss function. We also develop an evaluation reliability function to derive group weight. Finally, we provide a step-by-step numerical example of a loss function approach.     

勾股模糊偏好关系及其在群体决策中的应用

以区间模糊偏好关系(IVFPR)和直觉模糊偏好关系(IFPR)的理论框架为依据,将勾股模糊数(PFN)引入偏好关系中,定义勾股模糊偏好关系(PFPR)和加性一致性PFPR.然后,提出标准化勾股模糊权重向量(PFWV)的概念,并给出构造加性一致性PFPR的转换公式.为获取任意给定的PFPR的权重向量,建立以给定的PFPR与构造的加性一致性PFPR偏差最小为目标的优化模型.针对多个勾股模糊偏好关系的集结,利用能够有效处理极端值并满足关于序关系单调的勾股模糊加权二次(PFWQ)算子作为集结工具.进一步,联合PFWQ算子和目标优化模型提出一种群体决策方法.最后,通过案例分析表明所提出方法的实用性和可行性.