Boost-wise pre-loaded mixture of experts for classification tasks

A modified version of Boosted Mixture of Experts (BME) is presented in this paper. While previous related works, namely BME, attempt to improve the performance by incorporating complementary features of a hybrid combining framework, they have some drawback. Analyzing the problems of previous approaches has suggested several modifications that have led us to propose a new method called Boost-wise Pre-loaded Mixture of Experts (BPME). We present a modification in pre-loading (initialization) procedure of ME, which addresses previous problems and overcomes them by employing a two-stage pre-loading procedure. In this approach, both the error and confidence measures are used as the difficulty criteria in boost-wise partitioning of problem space.

     

Mixture of experts classification using a hierarchical mixture model

A three-level hierarchical mixture model for classification is presented that models the following data generation process: (1) the data are generated by a finite number of sources (clusters), and (2) the generation mechanism of each source assumes the existence of individual internal class-labeled sources (subclusters of the external cluster). The model estimates the posterior probability of class membership similar to a mixture of experts classifier. In order to learn the parameters of the model, we have developed a general training approach based on maximum likelihood that results in two efficient training algorithms. Compared to other classification mixture models, the proposed hierarchical model exhibits several advantages and provides improved classification performance as indicated by the experimental results.     

Electrocardiogram beat classification using empirical mode decomposition and multiclass directed acyclic graph support vector machine

In this paper, a classifier motivated from statistical learning theory, i.e., support vector machine, with a new approach based on multiclass directed acyclic graph has been proposed for classification of four types of electrocardiogram signals. The motivation for selecting Directed Acyclic Graph Support Vector Machine (DAGSVM) is to have more accurate classifier with less computational cost. Empirical mode decomposition and subsequently singular value decomposition have been used for computing the feature vector matrix. Further, fivefold cross-validation and particle swarm optimization have been used for optimal selection of SVM model parameters to improve the performance of DAGSVM. A comparison has been made between proposed algorithm and other two classifiers, i.e., K-Nearest Neighbor (KNN) and Artificial Neural Network (ANN). The DAGSVM has yielded an average accuracy of 98.96% against 95.83% and 96.66% for the KNN and the ANN, respectively. The results obtained clearly confirm the superiority of the DAGSVM approach over other classifiers.     

Joint target tracking and classification with particle filtering and mixture Kalman filtering using kinematic radar information

This paper considers the problem of joint maneuvering target tracking and classification. Based on recently proposed Monte Carlo techniques, a multiple model (MM) particle filter and a mixture Kalman filter (MKF) are designed for two-class identification of air targets: commercial and military aircraft. The classification task is carried out by processing radar measurements only, no class (feature) measurements are used. A speed likelihood function for each class is defined using a prior information about speed constraints. Class-dependent speed likelihoods are calculated through the state estimates of each class-dependent tracker. They are combined with the kinematic measurement likelihoods in order to improve the classification process. The two designed estimators are compared and evaluated over rather complex target scenarios. The results demonstrate the usefulness of the proposed scheme for the incorporation of additional speed information. Both filters illustrate the opportunity of the particle filtering and mixture Kalman filtering to incorporate constraints in a natural way, providing reliable tracking and correct classification. Future observations contain valuable information about the current state of the dynamic systems. In the framework of the MKF, an algorithm for delayed estimation is designed for improving the current modal state estimate. It is used as an additional, more reliable information in resolving complicated classification situations.     

Joint target tracking and classification via RFS-based multiple model filtering

Firstly, a multiple model extension of the random finite set (RFS)-based single-target Bayesian filtering (STBF), referred as MM-STBF, is presented to accommodate the possible target maneuvering behavior in a straightforward manner. This paper is concerned with joint target tracking and classification (JTC) which are closely coupled. In particular, we take into account extraneous target-originated measurements which were not modeled in the existing JTC algorithms. Therefore, the main contribution is that the paper derives a new JTC algorithm based on the MM-STBF, i.e., MM-STBF–JTC. The MM-STBF–JTC is an optimal Bayesian solution, which can simultaneously accommodate unknown data association, miss-detection, clutter and several measurements originated from a target. The MM-STBF–JTC can reduce to a traditional JTC algorithm under some assumptions. The simulation results are provided to demonstrate the tracking and classification performance of the MM-STBF–JTC algorithm.     

A modified mixture of experts network structure for ECG beats classification with diverse features

This paper illustrates the use of modified mixture of experts (MME) network structure to guide model selection for classification of electrocardiogram (ECG) beats with diverse features. The MME is a modular neural network architecture for supervised learning. Expectation-maximization (EM) algorithm was used for training the MME so that the learning process is decoupled in a manner that fits well with the modular structure. The wavelet coefficients and Lyapunov exponents of the ECG signals were computed and statistical features were calculated to depict their distribution. The statistical features, which were used for obtaining the diverse features of the ECG signals, were then input into the MME network structure for training and testing purposes. We explored the ability of designed and trained MME network structure, combined with wavelet preprocessing (computing wavelet coefficients) and nonlinear dynamics tools (computing Lyapunov exponents), to discriminate five types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat, partial epilepsy beat) obtained from the Physiobank database. The MME achieved accuracy rates which were higher than that of the mixture of experts (ME) and feedforward neural network models (multilayer perceptron neural network—MLPNN). The proposed MME approach can be useful in classifying long-term ECG signals for early detection of heart diseases/abnormalities.     

灰度极限脉冲耦合神经网络混合噪声滤波

针对图像中同时存在椒盐噪声和高斯噪声,提出一种基于灰度极限和脉冲耦合神经网络(PCNN)滤除混合噪声的新方法。首先,根据灰度极值定位出椒盐噪声点;其次,在滤波窗口中对椒盐噪声点进行均值滤波;然后,利用PCNN赋时矩阵定位出高斯噪声点;最后,自适应调整可变灰度步长,选择不同滤波方法滤除高斯噪声。实验结果表明提出的算法较常见的混合噪声滤波方法在主观滤波效果和客观评价指标峰值信噪比(PSNR)及信噪比改善因子(ISNR)两方面均有明显的优势。     

ECG beat classification by a novel hybrid neural network

This paper presents a novel hybrid neural network structure for the classification of the electrocardiogram (ECG) beats. Two feature extraction methods: Fourier and wavelet analyses for ECG beat classification are comparatively investigated in eight-dimensional feature space. ECG features are determined by dynamic programming according to the divergence value. Classification performance, training time and the number of nodes of the multi-layer perceptron (MLP), restricted Coulomb energy (RCE) and a novel hybrid neural network are comparatively presented. In order to increase the classification performance and to decrease the number of nodes, the novel hybrid structure is trained by the genetic algorithms (GAs). Ten types of ECG beats obtained from the MIT-BIH database and from a real-time ECG measurement system are classified with a success of 96% by using the hybrid structure.     

A multigradient algorithm using a mixture of experts architecture for land cover classification of multisensor images

An algorithm for supervised classification of multisensor images is proposed. The mixture of experts (ME) architecture with dynamic weight allocation is used for multiclass classification. Here the classification is treated as a maximum likelihood problem and the synaptic weights of the expert network and gating network are updated by a stochastic multigradient approach. Data from an optical sensor with four bands and a synthetic aperture radar (SAR) image of the same scene has been fused and classified. The algorithm is compared to some other advanced training algorithms in the literature for the same image data.     

Cost-sensitive boosting for classification of imbalanced data

Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. The significant difficulty and frequent occurrence of the class imbalance problem indicate the need for extra research efforts. The objective of this paper is to investigate meta-techniques applicable to most classifier learning algorithms, with the aim to advance the classification of imbalanced data. The AdaBoost algorithm is reported as a successful meta-technique for improving classification accuracy. The insight gained from a comprehensive analysis of the AdaBoost algorithm in terms of its advantages and shortcomings in tacking the class imbalance problem leads to the exploration of three cost-sensitive boosting algorithms, which are developed by introducing cost items into the learning framework of AdaBoost. Further analysis shows that one of the proposed algorithms tallies with the stagewise additive modelling in statistics to minimize the cost exponential loss. These boosting algorithms are also studied with respect to their weighting strategies towards different types of samples, and their effectiveness in identifying rare cases through experiments on several real world medical data sets, where the class imbalance problem prevails.     

Invariant texture classification for biomedical cell specimens via non-linear polar map filtering

A novel texture-based classification scheme for cell specimens that is robust over a range of orientation, scale and contrast values is proposed. We achieve this robustness by first segmenting the cell specimens and for each specimen, we find the largest ellipse that can be contained within it, and from this, we then construct an orientation and scale-invariant polar map. Non-linear filtering by normalized cross-correlation is then performed on the polar map to obtain contrast-invariant similarity maps. Local and global energy measures are finally extracted from these maps and classified using a support vector machine. Experimental results show that the proposed method achieves an average accuracy of about 97% in classifying six species of pollen, fungal and fern spores. In addition, every invariant property was validated through a series of experiments. Unlike conventional wavelet decomposition, Laws filtering and co-occurrence methods, our method shows a consistently high classification accuracy for all classes of cell specimens in an airspora dataset.     

A mixture of experts for classifying sleep apneas

This paper presents a novel approach for classifying sleep apneas into one of the three basic types: obstructive, central and mixed. The goal is to overcome the problems encountered in previous work and improve classification accuracy. The proposed model uses a new classification approach based on the characteristics that each type of apnea presents in different segments of the signal. The model is based on the error correcting output code and it is formed by a combination of artificial neural networks experts where their inputs are the coefficients obtained by a discrete wavelet decomposition applied to the raw samples of the apnea in the thoracic effort signal. The input coefficients received for each network were determined by a feature selection method (support vector machine recursive feature elimination). In order to train and test the systems, 120 events from six different patients were used. The true error rate was estimated using a 10-fold cross validation. The results presented in this work were averaged over 10 different simulations and a multiple comparison procedure was used for model selection. The mean test accuracy obtained was 90.27% ± 0.79, and the values for each class apnea were 94.62% (obstructive), 95.47% (central) and 90.45% (mixed). Up to the authors’ knowledge, the proposed classifier surpasses all previous results.     

A flexible probabilistic framework for large-margin mixture of experts

Mixture-of-Experts (MoE) enable learning highly nonlinear models by combining simple expert models. Each expert handles a small region of the data space, as dictated by the gating network which generates the (soft) assignment of input to the corresponding experts. Despite their flexibility and renewed interest lately, existing MoE constructions pose several difficulties during model training. Crucially, neither of the two popular gating networks used in MoE, namely the softmax gating network and hierarchical gating network (the latter used in the hierarchical mixture of experts), have efficient inference algorithms. The problem is further exacerbated if the experts do not have conjugate likelihood and lack a naturally probabilistic formulation (e.g., logistic regression or large-margin classifiers such as SVM). To address these issues, we develop novel inference algorithms with closed-form parameter updates, leveraging some of the recent advances in data augmentation techniques. We also present a novel probabilistic framework for MoE, consisting of a range of gating networks with efficient inference made possible through our proposed algorithms. We exploit this framework by using Bayesian linear SVMs as experts on various classification problems (which has a non-conjugate likelihood otherwise generally), providing our final model with attractive large-margin properties. We show that our models are significantly more efficient than other training algorithms for MoE while outperforming other traditional non-linear models like Kernel SVMs and Gaussian Processes on several benchmark datasets.

     

不平衡数据度量指标优化的提升分类方法

为提高不平衡数据的分类性能,提出了基于度量指标优化的不平衡数据Boosting算法。该算法结合不平衡数据分类性能度量标准和Boosting算法,使用不平衡数据分类性能度量指标代替原有误分率指标,分别采用带有权重的正类和负类召回率、F-measure和G-means指标对Boosting算法进行优化,按照不同的度量指标计算Alpha 值进行迭代,得到带有加权值的弱学习器组合,最后使用Boosting算法进行优化。经过实验验证,与带有权重的Boosting算法进行比较,该算法对一定数据集的AUC分类性能指标有一定提高,错误率有所下降,对F-measure和G-mean性能指标有一定的改善,说明该算法侧重提高正类分类性能,改善不平衡数据的整体分类性能。     

A review of boosting methods for imbalanced data classification

Recently, the problem of imbalanced data classification has drawn a significant amount of interest from academia, industry and government funding agencies. The fundamental issue with imbalanced data classification is the imbalanced data has posed a significant drawback of the performance of most standard learning algorithms, which assume or expect balanced class distribution or equal misclassification costs. Boosting is a meta-technique that is applicable to most learning algorithms. This paper gives a review of boosting methods for imbalanced data classification, denoted as IDBoosting (Imbalanced-data-boosting), where conventional learning algorithms can be integrated without further modifications. The main focus is on the intrinsic mechanisms without considering implementation detail. Existing methods are catalogued and each class is displayed in detail in terms of design criteria, typical algorithms and performance analysis. The essence of two IDBoosting methods is discovered followed by experimental evidence and useful reference point for future research are also given.     

RGBD co-saliency detection via multiple kernel boosting and fusion

Multimedia Tools and Applications - RGBD co-saliency detection, which aims at extracting common salient objects from a group of RGBD images with the additional depth information, has become an...     

基于改进K均值聚类生成匹配模板的心搏分类方法

为提高分析含大量数据的动态心电时的准确性和分析效率,提出了一种基于改进的K均值聚类生成心搏模板的匹配方法.使用K均值聚类和波形反混淆技术进行循环纠错,生成可变宽心搏模板、并建立心搏模板库.利用可变宽心搏模板和相关系数相结合的策略,对动态心电中心搏进行快速准确分类.实验方法经心率失常数据库MIT-BIT和ANMA/ANSI标准验证,分类结果总体准确率达98.06%,达到了心搏分类目标.     

LoCo: boosting for indoor location classification combining Wi-Fi and BLE

In recent years, there has been an explosion of services that leverage location to provide users novel and engaging experiences. However, many applications fail to realize their full potential because of limitations in current location technologies. Current frameworks work well outdoors but fare poorly indoors. In this paper, we present LoCo, a new framework that can provide highly accurate room-level indoor location. LoCo does not require users to carry specialized location hardware—it uses radios that are present in most contemporary devices and, combined with a boosting classification technique, provides a significant runtime performance improvement. We provide experiments that show the combined radio technique can achieve accuracy that improves on current state-of-the-art Wi-Fi-only techniques. LoCo is designed to be easily deployed within an environment and readily leveraged by application developers. We believe LoCo’s high accuracy and accessibility can drive a new wave of location-driven applications and services.     

QBoost: Predicting quantiles with boosting for regression and binary classification

In the framework of functional gradient descent/ascent, this paper proposes Quantile Boost (QBoost) algorithms which predict quantiles of the interested response for regression and binary classification. Quantile Boost Regression performs gradient descent in functional space to minimize the objective function used by quantile regression (QReg). In the classification scenario, the class label is defined via a hidden variable, and the quantiles of the class label are estimated by fitting the corresponding quantiles of the hidden variable. An equivalent form of the definition of quantile is introduced, whose smoothed version is employed as the objective function, and then maximized by functional gradient ascent to obtain the Quantile Boost Classification algorithm. Extensive experimentation and detailed analysis show that QBoost performs better than the original QReg and other alternatives for regression and binary classification. Furthermore, QBoost is capable of solving problems in high dimensional space and is more robust to noisy predictors.     

Rotation invariant curvelet based image retrieval & classification via Gaussian mixture model and co-occurrence features

Multimedia Tools and Applications - Demand for better retrieval methods continue to outstrip the capabilities of available technologies despite the rapid growth of new feature extraction...