Feature selection for high-dimensional machinery fault diagnosis data using multiple models and Radial Basis Function networks

The technique of machinery fault diagnosis has been greatly enhanced over recent years with the application of many pattern classification methods. However, these classification methods suffer from the “curse of dimensionality” when applied to high-dimensional fault diagnosis data. In order to solve the problem, this paper proposes a hybrid model which combines multiple feature selection models to select the most significant input features from all potentially relevant features. Among the models, eight filter models are used to pre-rank the candidate features. They include data variance, Pearson correlation coefficient, the Relief algorithm, Fisher score, class separability, chi-squared, information gain and gain ratio. These variable ranking models measure features from various perspectives, and lead to different ranking results. Based on the effect of the ranking results on the Radial Basis Function (RBF) classification, a weighted voting scheme is then introduced to re-rank features. Furthermore, two wrapper models, a Binary Search (BS) model and a Sequential Backward Search (SBS) model are utilized to minimize the number of relevant features. To demonstrate the potential for applying the method to machinery fault diagnosis, two case studies are discussed. The experiment results support the conclusion that this method is useful for revealing fault-related frequency features.     

图像分类卷积神经网络的特征选择模型压缩方法

深度卷积神经网络(convolutional neural networks, CNN)作为特征提取器(feature extractor, CNN--FE)已被广泛应用于许多领域并获得显著成功. 根据研究评测可知CNN--FE具有大量参数, 这大大限制了CNN--FE在如智能手机这样的内存有限的设备上的应用. 本文以AlexNet卷积神经网络特征提取器为研究对象, 面向图像分类问题, 在保持图像分类性能几乎不变的情况下减少CNN--FE模型参数量. 通过对AlexNet各层参数分布的详细分析, 作者发现其全连接层包含了大约99%的模型参数, 在图像分类类别较少的情况, AlexNet提取的特征存在冗余. 因此, 将CNN--FE模型压缩问题转化为深度特征选择问题, 联合考虑分类准确率和压缩率, 本文提出了一种新的基于互信息量的特征选择方法, 实现CNN--FE模型压缩. 在公开场景分类数据库以及自建的无线胶囊内窥镜(wireless capsule endoscope, WCE)气泡图片数据库上进行图像分类实验. 结果表明本文提出的CNN--FE模型压缩方法减少了约83%的AlexNet模型参数且其分类准确率几乎保持不变.     

Neuro-logistic Models Based on Evolutionary Generalized Radial Basis Function for the Microarray Gene Expression Classification Problem

Gene expression detection is a key bioinformatic problem which has been tackled as a classification problem of microarray gene expression, obtained by the light reflection analysis of genomic material. A typical microarray dataset may contain thousands of genes but only a small number of patterns (often less than two hundred). When the dataset presents these kinds of characteristics, state-of-the-art classification models show a high lack of performance. A two-stage algorithm has been proposed to successfully address the problem of microarray classification. In the first stage, two filter algorithms identify salient expression genes from thousands of genes. In the second stage, the proposed methodology is performed using selected gene subsets as new input variables. The methodology proposed is composed of a combination of Logistic Regression (LR) and Evolutionary Generalized Radial Basis Function (EGRBF) neural networks which have shown to be highly accurate in previous research in the modeling of high-dimensional patterns. Finally, the results obtained are contrasted with nonparametric statistical tests and confirm good synergy between EGRBF and LR models.     

Improved Complex-valued Radial Basis Function (ICRBF) neural networks on multiple crack identification

This paper introduces a new class of neural networks in complex space called Complex-valued Radial Basis Function (CRBF) neural networks and also an improved version of CRBF called Improved Complex-valued Radial Basis Function (ICRBF) neural networks. They are used for multiple crack identification in a cantilever beam in the frequency domain. The novelty of the paper is that, these complex-valued neural networks are first applied on inverse problems (damage identification) which come under the category of function approximation. The conventional CRBF network was used in the first stage of ICRBF network and in the second stage a reduced search space moving technique was employed for accurate crack identification. The effectiveness of proposed ICRBF neural network was studied first on a single crack identification problem and then applied to a more challenging problem of multiple crack identification in a cantilever beam with zero noise as well as 5% noise polluted signals. The results proved that, the proposed ICRBF and real-valued Improved RBF (IRBF) neural networks have identified the single and multiple cracks with less than 1% absolute mean percentage error as compared to conventional CRBF and RBF neural networks, mainly because of their second stage reduced search space moving technique. It appears that IRBF neural network is a good compromise considering all factors like accuracy, simplicity and computational effort.     

Stock market prediction using artificial neural networks with optimal feature transformation

This paper compares a feature transformation method using a genetic algorithm (GA) with two conventional methods for artificial neural networks (ANNs). In this study, the GA is incorporated to improve the learning and generalizability of ANNs for stock market prediction. Daily predictions are conducted and prediction accuracy is measured. In this study, three feature transformation methods for ANNs are compared. Comparison of the results achieved by a feature transformation method using the GA to the other two feature transformation methods shows that the performance of the proposed model is better. Experimental results show that the proposed approach reduces the dimensionality of the feature space and decreases irrelevant factors for stock market prediction.     

Factorized weight interaction neural networks for sparse feature prediction

Non-contiguous and categorical sparse feature data are widely existed on the Internet. To build a machine learning system with these data, it is important to properly model the interaction among features. In this paper, we propose a factorized weight interaction neural network (INN) with a new network structure called weight-interaction layer to learn patterns from feature interactions and factorized weight parameters of each feature interaction. The proposed INN can greatly reduce the dimension of sparse data via the weight-interaction layer, while the multi-layer neural network can be used to capture high-order feature latent patterns. Our experimental results on two real datasets show that the proposed method is able to effectively improve the prediction accuracy and generalization performance of the model, and consistently outperform related methods to be compared.

     

Combining neural networks and semantic feature space for email classification

Email is one of the most ubiquitous and pervasive applications used on a daily basis by millions of people worldwide, individuals and organizations more and more rely on the emails to communicate and share information and knowledge. However, the increase in email users has resulted in a dramatic increase in spam emails during the past few years. It is becoming a big challenge to process and manage the emails efficiently for and individuals and organizations. This paper proposes new email classification models using a linear neural network trained by perceptron learning algorithm and a nonlinear neural network trained by back-propagation learning algorithm. An efficient semantic feature space (SFS) method is introduced in these classification models. The traditional back-propagation neural network (BPNN) has slow learning speed and is prone to trap into a local minimum, so the modified back-propagation neural network (MBPNN) is presented to overcome these limitations. The vector space model based email classification system suffers from a large number of features and ambiguity in the meaning of terms, which will lead to sparse and noisy feature space. So we use the SFS to convert the original sparse and noisy feature space to a semantically richer feature space, which will helps to accelerate the learning speed. The experiments are conducted based on different training set size and extracted feature size. Experimental results show that the models using MBPNN outperform the traditional BPNN, and the use of SFS can greatly reduce the feature dimensionality and improve email classification performance.     

Statistical inference in a redesigned Radial Basis Function neural network

A Hybrid Learning Process method was fitted into a RBF. The resulting redesigned RBF intends to show how to test if the statistical assumptions are fulfilled and to apply statistical inference to the redesigned RBFNN bearing in mind that it allows to determine the relationship between a response (to a process) and one or more independent variables, testing how much each factor contributes to the total variation of the response is also feasible. The results show that statistical methods such as inference, Residual Analysis, and statistical metrics are all good alternatives and excellent methods for validation of the effectiveness of the Neural Network models. The foremost conclusion is that the resulting redesigned Radial Basis Function improved the accuracy of the model after using a Hybrid Learning Process; moreover, the new model also validates the statistical assumptions for using statistical inference and statistical analysis, satisfying the assumptions required for ANOVA to determine the statistical significance and the relationship between variables.     

Audiovisual emotion recognition using ANOVA feature selection method and multi-classifier neural networks

To make human–computer interaction more naturally and friendly, computers must enjoy the ability to understand human’s affective states the same way as human does. There are many modals such as face, body gesture and speech that people use to express their feelings. In this study, we simulate human perception of emotion through combining emotion-related information using facial expression and speech. Speech emotion recognition system is based on prosody features, mel-frequency cepstral coefficients (a representation of the short-term power spectrum of a sound) and facial expression recognition based on integrated time motion image and quantized image matrix, which can be seen as an extension to temporal templates. Experimental results showed that using the hybrid features and decision-level fusion improves the outcome of unimodal systems. This method can improve the recognition rate by about 15 % with respect to the speech unimodal system and by about 30 % with respect to the facial expression system. By using the proposed multi-classifier system that is an improved hybrid system, recognition rate would increase up to 7.5 % over the hybrid features and decision-level fusion with RBF, up to 22.7 % over the speech-based system and up to 38 % over the facial expression-based system.     

Convolutional neural networks combined with feature selection for radio-frequency fingerprinting

Radio-frequency fingerprinting is a technique for the authentication and identification of wireless devices using their intrinsic physical features and an analysis of the digitized signal collected during transmission. The technique is based on the fact that the unique physical features of the devices generate discriminating features in the transmitted signal, which can then be analyzed using signal-processing and machine-learning algorithms. Deep learning and more specifically convolutional neural networks (CNNs) have been successfully applied to the problem of radio-frequency fingerprinting using a spectral domain representation of the signal. A potential problem is the large size of the data to be processed, because this size impacts on the processing time during the application of the CNN. We propose an approach to addressing this problem, based on dimensionality reduction using feature-selection algorithms before the spectrum domain representation is given as an input to the CNN. The approach is applied to two public data sets of radio-frequency devices using different feature-selection algorithms for different values of the signal-to-noise ratio. The results show that the approach is able to achieve not only a shorter processing time; it also provides a superior classification performance in comparison to the direct application of CNNs.     

An expert model for self-care problems classification using probabilistic neural network and feature selection approach

Self-care problems classification is one of the important challenges for occupational therapists. Extent and variety of disorders make the self-care problems classification process complex and time-consuming. To overcome this challenge, an expert model is proposed innovatively in this research. The proposed model is based on Probabilistic Neural Network (PNN) and Genetic Algorithm (GA) for classifying self-care problems of children with physical and motor disability. In this model, PNN is employed as a classifier and GA is applied for feature selection. The PNN is trained by using a standard ICF-CY dataset. Based on ICF-CY, occupational therapists must evaluate many features to diagnose self-care problems. According to the experiences of occupational therapists, these features have different effects on classification. Hence, GA is employed to select relevant and important features in self-care problems classification. Since the classification rules are important for occupational therapists, the self-care problems classification rules are extracted additionally by using the CART algorithm. The experimental results show that by using the feature selection algorithm, the accuracy and time complexity of classification are improved in comparison to other models. The proposed model can classify self-care problems of children with 94.28% accuracy by using only 16.5% of all features.     

Advances in evolutionary feature selection neural networks with co-evolution learning

Training neural networks is a complex task provided that many algorithms are combined to find best solutions to the classification problem. In this work, we point out the evolutionary computing to minimize a neural configuration. For this purpose, a distribution estimation framework is performed to select relevant features, which lead to classification accuracy with a lower complexity in computational time. Primarily, a pruning strategy-based score function is applied to decide the network relevance in the genetic population. Since the complexity of the network (connections, weights, and biases) is most important, the cooling state of the system will strongly relate to the entropy as a minimization function to reach the desired solution. Also, the framework proposes coevolution learning (with discrete and continuous representations) to improve the behavior of the evolutionary neural learning. The results obtained after simulations show that the proposed work is a promising way to extend its usability to other classes of neural networks.     

Time series prediction using PSO-optimized neural network and hybrid feature selection algorithm for IEEE load data

Artificial neural networks have been widely used in time series prediction. In this paper, multi-layer feedforward neural networks with optimized structures, using particle swarm optimization (PSO) algorithm, are used for hourly load prediction based on load time series of IEEE Reliability Test System. To have a small and appropriate feature subset, a hybrid method is used for feature selection in this paper. This hybrid method uses the combination of genetic algorithm (GA) and ant colony optimization (ACO) algorithm. The season, day of the week, time of the day and history load are considered as load influencing factors in this study based on the mentioned standard load dataset. The optimized number of neurons in the hidden layers of multi-layer perceptron (MLP) is determined using PSO algorithm. Experimental results show that the proposed hybrid model offers superior performance, in terms of mean absolute percentage error (MAPE), in time series prediction as compared to some recent researches in this field.     

Object-based feature selection using class-pair separability for high-resolution image classification

ABSTRACT

With the increasing popularity of object-based image analysis (OBIA) since 2006, numerous classification and mapping tasks were reported to benefit from this evolving paradigm. In these studies, segments are firstly created, followed by classification based on segment-level information. However, the feature space formed by segment-level feature variables can be very large and complex, posing challenges to obtaining satisfactory classification performance. Accordingly, this work attempts to develop a new feature selection approach for segment-level features. Based on the principle of class-pair separability, the segment-level features are grouped according to their types. For each group, the contribution of each segment-level feature to the separation of a pair of classes is quantified. With the information of all feature groups and class pairs, the separability ranking and appearance frequency are considered to compute importance score for each feature. Higher importance score means larger appropriateness to select a feature. By using two Gaofen-2 multi-spectral images, the proposed method is validated. The experimental results show the advantages of the proposed technique over some state-of-the-art feature selection approaches: (1) it can better reduce the number of segment-level features and effectively avoid redundant information; (2) the feature subset obtained by the proposed scheme has good potential to improve classification accuracy.     

Evolutionary feature subsets selection based on interaction information for high dimensional imbalanced data classification

With the advent of technology in various scientific fields, high dimensional data are becoming abundant. A general approach to tackle the resulting challenges is to reduce data dimensionality through feature selection. Traditional feature selection approaches concentrate on selecting relevant features and ignoring irrelevant or redundant ones. However, most of these approaches neglect feature interactions. On the other hand, some datasets have imbalanced classes, which may result in biases towards the majority class. The main goal of this paper is to propose a novel feature selection method based on the interaction information (II) to provide higher level interaction analysis and improve the search procedure in the feature space. In this regard, an evolutionary feature subset selection algorithm based on interaction information is proposed, which consists of three stages. At the first stage, candidate features and candidate feature pairs are identified using traditional feature weighting approaches such as symmetric uncertainty (SU) and bivariate interaction information. In the second phase, candidate feature subsets are formed and evaluated using multivariate interaction information. Finally, the best candidate feature subsets are selected using dominant/dominated relationships. The proposed algorithm is compared with some other feature selection algorithms including mRMR, WJMI, IWFS, IGFS, DCSF, IWFS, K_OFSD, WFLNS, Information Gain and ReliefF in terms of the number of selected features, classification accuracy, F-measure and algorithm stability using three different classifiers, namely KNN, NB, and CART. The results justify the improvement of classification accuracy and the robustness of the proposed method in comparison with the other approaches.     

Lung cancer classification using neural networks for CT images

Early detection of cancer is the most promising way to enhance a patient's chance for survival. This paper presents a computer aided classification method in computed tomography (CT) images of lungs developed using artificial neural network. The entire lung is segmented from the CT images and the parameters are calculated from the segmented image. The statistical parameters like mean, standard deviation, skewness, kurtosis, fifth central moment and sixth central moment are used for classification. The classification process is done by feed forward and feed forward back propagation neural networks. Compared to feed forward networks the feed forward back propagation network gives better classification. The parameter skewness gives the maximum classification accuracy. Among the already available thirteen training functions of back propagation neural network, the Traingdx function gives the maximum classification accuracy of 91.1%. Two new training functions are proposed in this paper. The results show that the proposed training function 1 gives an accuracy of 93.3%, specificity of 100% and sensitivity of 91.4% and a mean square error of 0.998. The proposed training function 2 gives a classification accuracy of 93.3% and minimum mean square error of 0.0942.     

Cancer classification using ensemble of neural networks with multiple significant gene subsets

Molecular level diagnostics based on microarray technologies can offer the methodology of precise, objective, and systematic cancer classification. Genome-wide expression patterns generally consist of thousands of genes. It is desirable to extract some significant genes for accurate diagnosis of cancer because not all genes are associated with a cancer. In this paper, we have used representative gene vectors that are highly discriminatory for cancer classes and extracted multiple significant gene subsets based on those representative vectors respectively. Also, an ensemble of neural networks learned from the multiple significant gene subsets is proposed to classify a sample into one of several cancer classes. The performance of the proposed method is systematically evaluated using three different cancer types: Leukemia, colon, and B-cell lymphoma.     

Effective fine-grained feature extraction and classification of solid materials using hybrid region convolutional neural networks

Multimedia Tools and Applications - Difference between similar feature points is presented in the fine-grained classification, which depends on discriminative in extremely localized regions. Hence,...     

Joint damage identification using Improved Radial Basis Function (IRBF) networks in frequency and time domain

In this paper, a novel two-stage Improved Radial Basis Function (IRBF) neural network technique is proposed to predict the joint damage of a fifty member frame structure with semi-rigid connections in both frequency and time domain. The effective input patterns as normalized design signature indices (NDSIs) in frequency domain and acceleration responses in time domain are simulated numerically from finite element analysis (FEA) by considering different levels of damage severity using Latin hypercube sampling (LHS) technique. The conventional RBF network is used in the first stage of IRBF network and in the second stage reduced search space moving technique is employed for accurate prediction with less than 3% error. The numerical simulation of the substructural joint damage identification of a fifty member frame structure with and without addition of 5% Gaussian random noise to the input patterns is presented and compared with conventional CPN–BPN hybrid method. The two-stage IRBF method is found to be superior in accuracy to conventional hybrid methods as well as to conventional RBF method. An important benefit of the proposed novel IRBF method is the significant reduction in the computational time with good accuracy of joint damage identification.     

Power transformer differential protection using neural network Principal Component Analysis and Radial Basis Function Neural Network

Many methods have been used to discriminate magnetizing inrush from internal faults in power transformers. Most of them follow a deterministic approach, i.e. they rely on an index and fixed threshold. This article proposes two approaches (i.e. NNPCA and RBFNN) for power transformer differential protection and address the challenging task of detecting magnetizing inrush from internal fault. These approaches based on the pattern recognition technique. In the proposed algorithm, the Neural Network Principal Component Analysis (NNPCA) and Radial Basis Function Neural Network (RBFNN) are used as a classifier. The principal component analysis is used to preprocess the data from power system in order to eliminate redundant information and enhance hidden pattern of differential current to discriminate between internal faults from inrush and over-excitation condition. The presented algorithm also makes use of ratio of voltage-to-frequency and amplitude of differential current for detection transformer operating condition. For both proposed cases, optimal number of neurons has been considered in the neural network architectures and the effect of hidden layer neurons on the classification accuracy is analyzed. A comparison among the performance of the FFBPNN (Feed Forward Back Propagation Neural Network), NNPCA, RBFNN based classifiers and with the conventional harmonic restraint method based on Discrete Fourier Transform (DFT) method is presented in distinguishing between magnetizing inrush and internal fault condition of power transformer. The algorithm is evaluated using simulation performed with PSCAD/EMTDC and MATLAB. The results confirm that the RBFNN is faster, stable and more reliable recognition of transformer inrush and internal fault condition.