Unsupervised feature selection using swarm intelligence and consensus clustering for automatic fault detection and diagnosis in Heating Ventilation and Air Conditioning systems

Various sensory and control signals in a Heating Ventilation and Air Conditioning (HVAC) system are closely interrelated which give rise to severe redundancies between original signals. These redundancies may cripple the generalization capability of an automatic fault detection and diagnosis (AFDD) algorithm. This paper proposes an unsupervised feature selection approach and its application to AFDD in a HVAC system. Using Ensemble Rapid Centroid Estimation (ERCE), the important features are automatically selected from original measurements based on the relative entropy between the low- and high-frequency features. The materials used is the experimental HVAC fault data from the ASHRAE-1312-RP datasets containing a total of 49 days of various types of faults and corresponding severity. The features selected using ERCE (Median normalized mutual information (NMI) = 0.019) achieved the least redundancies compared to those selected using manual selection (Median NMI = 0.0199) Complete Linkage (Median NMI = 0.1305), Evidence Accumulation K-means (Median NMI = 0.04) and Weighted Evidence Accumulation K-means (Median NMI = 0.048). The effectiveness of the feature selection method is further investigated using two well-established time-sequence classification algorithms: (a) Nonlinear Auto-Regressive Neural Network with eXogenous inputs and distributed time delays (NARX-TDNN); and (b) Hidden Markov Models (HMM); where weighted average sensitivity and specificity of: (a) higher than 99% and 96% for NARX-TDNN; and (b) higher than 98% and 86% for HMM is observed. The proposed feature selection algorithm could potentially be applied to other model-based systems to improve the fault detection performance.     

基于SAE和GNDO-SVM的脑电信号情绪识别

情感计算是现代人机交互中的关键问题, 随着人工智能的发展, 基于脑电信号(electroencephalogram, EEG)的情绪识别已经成为重要的研究方向. 为了提高情绪识别的分类精度, 本研究引入堆叠自动编码器(stacked auto-encoder, SAE)对EEG多通道信号进行深度特征提取, 并提出一种基于广义正态分布优化的支持向量机(generalized normal distribution optimization based support vector machine, GNDO-SVM)情绪识别模型. 实验结果表明, 与基于遗传算法、粒子群算法和麻雀搜索算法优化的支持向量机模型相比, 所提出的GNDO-SVM模型具有更优的分类性能, 基于SAE深度特征的情感识别准确率达到了90.94%, 表明SAE能够有效地挖掘EEG信号不同通道间的深度相关性信息. 因此, 利用SAE深度特征结合GNDO-SVM模型可以有效地实现EEG信号的情绪识别.     

Spoken emotion recognition through optimum-path forest classification using glottal features

A new method for the recognition of spoken emotions is presented based on features of the glottal airflow signal. Its effectiveness is tested on the new optimum path classifier (OPF) as well as on six other previously established classification methods that included the Gaussian mixture model (GMM), support vector machine (SVM), artificial neural networks – multi layer perceptron (ANN-MLP), k-nearest neighbor rule (k-NN), Bayesian classifier (BC) and the C4.5 decision tree. The speech database used in this work was collected in an anechoic environment with ten speakers (5 M and 5 F) each speaking ten sentences in four different emotions: Happy, Angry, Sad, and Neutral. The glottal waveform was extracted from fluent speech via inverse filtering. The investigated features included the glottal symmetry and MFCC vectors of various lengths both for the glottal and the corresponding speech signal. Experimental results indicate that best performance is obtained for the glottal-only features with SVM and OPF generally providing the highest recognition rates, while for GMM or the combination of glottal and speech features performance was relatively inferior. For this text dependent, multi speaker task the top performing classifiers achieved perfect recognition rates for the case of 6th order glottal MFCCs.     

Fusion of voice signal information for detection of mild laryngeal pathology

Detection of mild laryngeal disorders using acoustic parameters of human voice is the main objective in this study. Observations of sustained phonation (audio recordings of vocalized /a/) are labeled by clinical diagnosis and rated by severity (from 0 to 3). Research is exclusively constrained to healthy (severity 0) and mildly pathological (severity 1) cases – two the most difficult classes to distinguish between.Comprehensive voice signal characterization and information fusion constitute the approach adopted here. Characterization is obtained through diverse feature set, containing 26 feature subsets of varying size, extracted from the voice signal. Usefulness of feature-level and decision-level fusion is explored using support vector machine (SVM) and random forest (RF) as basic classifiers. For both types of fusion we also investigate the influence of feature selection on model accuracy. To improve the decision-level fusion we introduce a simple unsupervised technique for ensemble design, which is based on partitioning the feature set by k-means clustering, where the parameter k controls the size and diversity of the prospective ensemble.All types of the fusion resulted in an evident improvement over the best individual feature subset. However, none of the types, including fusion setups comprising feature selection, proved to be significantly superior over the rest. The proposed ensemble design by feature set decomposition discernibly enhanced decision-level and significantly outperformed feature-level fusion. Ensemble of RF classifiers, induced from a cluster-based partitioning of the feature set, achieved equal error rate of 13.1 ± 1.8% in the detection of mildly pathological larynx. This is a very encouraging result, considering that detection of mild laryngeal disorder is a more challenging task than a common discrimination between healthy and a wide spectrum of pathological cases.     

Bio-inspired feature selection to select informative image features for determining water content of cultured Sunagoke moss

One of the primary determinants of Sunagoke moss Rachomitrium japonicum growth is water availability. There is need to develop a non-destructive method for sensing water content of cultured Sunagoke moss to realize automation and precision irrigation in a close bio-production systems. Machine vision can be utilized as non-destructive sensing to recognize changes in some kind of features that describe the water conditions from the appearance of wilting Sunagoke moss. The goal of this study is to propose and investigate bio-inspired algorithms i.e. Neural-Ant Colony Optimization, Neural-Genetic Algorithms, Neural-Simulated Annealing and Neural-Discrete Particle Swarm Optimization to find the most significant sets of image features suitable for predicting water content of cultured Sunagoke moss. Image features consist of 8 colour features, three morphological features and 90 textural features (RGB, HSV, HSL colour co-occurrence matrix and gray level co-occurrence matrix textural features). Each colour space of textural features consist of energy, entropy, contrast, homogeneity, inverse difference moment, correlation, sum mean, variance, cluster tendency and maximum probability. The specificity of this problem is that we are not looking for single image feature but several associations of image features that may be involved in determining water content of cultured Sunagoke moss. All feature selection models showed that prediction performance is getting better through all the iterations. It indicates that all models are effective. Neural-Ant Colony Optimization had the best performance as a feature selection technique. The minimum average prediction mean square error (MSE) achieved was 1.75 × 10^?3. There is significant improvement between method using feature selection and method without feature selection.     

An expert system to predict protein thermostability using decision tree

Protein thermostability information is closely linked to commercial production of many biomaterials. Recent developments have shown that amino acid composition, special sequence patterns and hydrogen bonds, disulfide bonds, salt bridges and so on are of considerable importance to thermostability. In this study, we present a system to integrate these various factors that predict protein thermostability. In this study, the features of proteins in the PGTdb are analyzed. We consider both structure and sequence features and correlation coefficients are incorporated into the feature selection algorithm. Machine learning algorithms are then used to develop identification systems and performances between the different algorithms are compared. In this research, two features, (E + F + M + R)/residue and charged/non-charged, are found to be critical to the thermostability of proteins. Although the sequence and structural models achieve a higher accuracy, sequence-only models provides sufficient accuracy for sequence-only thermostability prediction.     

An improved swarm optimized functional link artificial neural network (ISO-FLANN) for classification

Multilayer perceptron (MLP) (trained with back propagation learning algorithm) takes large computational time. The complexity of the network increases as the number of layers and number of nodes in layers increases. Further, it is also very difficult to decide the number of nodes in a layer and the number of layers in the network required for solving a problem a priori. In this paper an improved particle swarm optimization (IPSO) is used to train the functional link artificial neural network (FLANN) for classification and we name it ISO-FLANN. In contrast to MLP, FLANN has less architectural complexity, easier to train, and more insight may be gained in the classification problem. Further, we rely on global classification capabilities of IPSO to explore the entire weight space, which is plagued by a host of local optima. Using the functionally expanded features; FLANN overcomes the non-linear nature of problems. We believe that the combined efforts of FLANN and IPSO (IPSO + FLANN = ISO ? FLANN) by harnessing their best attributes can give rise to a robust classifier. An extensive simulation study is presented to show the effectiveness of proposed classifier. Results are compared with MLP, support vector machine(SVM) with radial basis function (RBF) kernel, FLANN with gradiend descent learning and fuzzy swarm net (FSN).     

A novel memetic algorithm for discovering knowledge in binary and multi class predictions based on support vector machine

In classification, every feature of the data set is an important contributor towards prediction accuracy and affects the model building cost. To extract the priority features for prediction, a suitable feature selector is schemed. This paper proposes a novel memetic based feature selection model named Shapely Value Embedded Genetic Algorithm (SVEGA). The relevance of each feature towards prediction is measured by assembling genetic algorithms with shapely value measures retrieved from SVEGA. The obtained results are then evaluated using Support Vector Machine (SVM) with different kernel configurations on 11 + 11 benchmark datasets (both binary class and multi class). Eventually, a contrasting analysis is done between SVEGA-SVM and other existing feature selection models. The experimental results with the proposed setup provides robust outcome; hence proving it to be an efficient approach for discovering knowledge via feature selection with improved classification accuracy compared to conventional methods.     

A novel method for automatic modulation recognition

Automatic recognition of the digital modulation plays an important role in various applications. This paper investigates the design of an accurate system for recognition of digital modulations. First, it is introduced an efficient pattern recognition system that includes two main modules: the feature extraction module and the classifier module. Feature extraction module extracts a suitable combination of the higher order moments up to eighth, higher order cumulants up to eighth and instantaneous characteristics of digital modulations. These combinations of the features are applied for the first time in this area. In the classifier module, two important classes of supervised classifiers, i.e., multi-layer perceptron (MLP) neural network and hierarchical multi-class support vector machine based classifier are investigated. By experimental study, we choose the best classifier for recognition of the considered modulations. Then, we propose a hybrid heuristic recognition system that an optimization module is added to improve the generalization performance of the classifier. In this module we have used a new optimization algorithm called Bees Algorithm. This module optimizes the classifier design by searching for the best value of the parameters that tune its discriminant function, and upstream by looking for the best subset of features that feed the classifier. Simulation results show that the proposed hybrid intelligent technique has very high recognition accuracy even at low levels of SNR with a little number of the features.     

Human activity recognition with smartphone sensors using deep learning neural networks

Human activities are inherently translation invariant and hierarchical. Human activity recognition (HAR), a field that has garnered a lot of attention in recent years due to its high demand in various application domains, makes use of time-series sensor data to infer activities. In this paper, a deep convolutional neural network (convnet) is proposed to perform efficient and effective HAR using smartphone sensors by exploiting the inherent characteristics of activities and 1D time-series signals, at the same time providing a way to automatically and data-adaptively extract robust features from raw data. Experiments show that convnets indeed derive relevant and more complex features with every additional layer, although difference of feature complexity level decreases with every additional layer. A wider time span of temporal local correlation can be exploited (1 × 9–1 × 14) and a low pooling size (1 × 2–1 × 3) is shown to be beneficial. Convnets also achieved an almost perfect classification on moving activities, especially very similar ones which were previously perceived to be very difficult to classify. Lastly, convnets outperform other state-of-the-art data mining techniques in HAR for the benchmark dataset collected from 30 volunteer subjects, achieving an overall performance of 94.79% on the test set with raw sensor data, and 95.75% with additional information of temporal fast Fourier transform of the HAR data set.     

An expert system for automated recognition of patients with obstructive sleep apnea using electrocardiogram recordings

Obstructive sleep apnea (OSA) is a highly prevalent sleep disorder. The traditional diagnosis methods of the disorder are cumbersome and expensive. The ability to automatically identify OSA from electrocardiogram (ECG) recordings is important for clinical diagnosis and treatment. In this study, we proposed an expert system based on discrete wavelet transform (DWT), fast-Fourier transform (FFT) and least squares support vector machine (LS-SVM) for the automatic recognition of patients with OSA from nocturnal ECG recordings. Thirty ECG recordings collected from normal subjects and subjects with sleep apnea, each of approximately 8 h in duration, were used throughout the study. The proposed OSA recognition system comprises three stages. In the first stage, an algorithm based on DWT was used to analyze ECG recordings for the detection of heart rate variability (HRV) and ECG-derived respiration (EDR) changes. In the second stage, an FFT based power spectral density (PSD) method was used for feature extraction from HRV and EDR changes. Then, a hill-climbing feature selection algorithm was used to identify the best features that improve classification performance. In the third stage, the obtained features were used as input patterns of the LS-SVM classifier. Using the cross-validation method, the accuracy of the developed system was found to be 100% for using a subset of selected combination of HRV and EDR features. The results confirmed that the proposed expert system has potential for recognition of patients with suspected OSA by using ECG recordings.     

Automatic identification of discourse markers in dialogues: An in-depth study of like and well

The lexical items like and well can serve as discourse markers (DMs), but can also play numerous other roles, such as verb or adverb. Identifying the occurrences that function as DMs is an important step for language understanding by computers. In this study, automatic classifiers using lexical, prosodic/positional and sociolinguistic features are trained over transcribed dialogues, manually annotated with DM information. The resulting classifiers improve state-of-the-art performance of DM identification, at about 90% recall and 79% precision for like (84.5% accuracy, κ = 0.69), and 99% recall and 98% precision for well (97.5% accuracy, κ = 0.88). Automatic feature analysis shows that lexical collocations are the most reliable indicators, followed by prosodic/positional features, while sociolinguistic features are marginally useful for the identification of DM like and not useful for well. The differentiated processing of each type of DM improves classification accuracy, suggesting that these types should be treated individually.     

Genetic programming for epileptic pattern recognition in electroencephalographic signals

This paper reports how the genetic programming paradigm, in conjunction with pattern recognition principles, can be used to evolve classifiers capable of recognizing epileptic patterns in human electroencephalographic signals. The procedure for feature extraction from the raw signal is detailed, as well as the genetic programming system that properly selects the features and evolves the classifiers. Based on the data sets used, two different epileptic patterns were detected: 3 Hz spike-and-slow-wave-complex (SASWC) and spike-or-sharp-wave (SOSW). After training, classifiers for both patterns were tested with unseen instances, and achieved sensibility = 1.00 and specificity = 0.93 for SASWC patterns, and sensibility = 0.94 and specificity = 0.89 for SOSW patterns. Results are very promising and suggest that the methodology presented can be applied to other pattern recognition tasks in complex signals.     

Multi-objective Grey Wolf Optimizer for improved cervix lesion classification

Cervical cancer is one of the vital and most frequent cancers, but can be cured effectively if diagnosed in the early stage. This is a novel effort towards effective characterization of cervix lesions from contrast enhanced CT-Scan images to provide a reliable and objective discrimination between benign and malignant lesions. Performance of such classification models mostly depends on features used to represent samples in a training dataset. Selection of optimal feature subset here is NP-hard; where, randomized algorithms do better. In this paper, Grey Wolf Optimizer (GWO), which is a population based meta-heuristic inspired by the leadership hierarchy and hunting mechanism of grey wolves has been utilized for feature selection. The traditional GWO is applicable for continuous single objective optimization problems. Since, feature selection is inherently multi-objective; this paper proposes two different approaches for multi-objective binary GWO algorithms. One is a scalarized approach to multi-objective GWO (MOGWO) and the other is a Non-dominated Sorting based GWO (NSGWO). These are used for wrapper based feature selection that selects optimal textural feature subset for improved classification of cervix lesions. For experiments, contrast enhanced CT-Scan (CECT) images of 62 patients have been used, where all lesions had been recommended for surgical biopsy by specialist. Gray-level co-occurrence matrix based texture features are extracted from two-level decomposition of wavelet coefficients of cervix regions extracted from CECT images. The results of proposed approaches are compared with mostly used meta-heuristics such as genetic algorithm (GA) and firefly algorithm (FA) for multi-objective optimization. With better diversification and intensification, GWO obtains Pareto solutions, which dominate the solutions obtained by GA and FA when assessed on the utilized cervix lesion cases. Cervix lesions are up to 91% accurately classified as benign and malignant with only five features selected by NSGWO. A two-tailed t-test was conducted by hypothesizing the mean F-score obtained by the proposed NSGWO method at significance level = 0.05. This confirms that NSGWO performs significantly better than other methods for the real cervix lesion dataset in hand. Further experiments were conducted on high dimensional microarray gene expression datasets collected online. The results demonstrate that the proposed method performs significantly better than other methods selecting relevant genes for high-dimensional, multi-category cancer diagnosis with an average of 12.82% improvement in F-score value.     

Consumer neuroscience: Assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking

Application of neuroscience methods to analyze and understand human behavior related to markets and marketing exchange has recently gained research attention. The basic aim is to guide design and presentation of products to optimize them to be as compatible as possible with consumer preferences. This paper investigates physiological decision processes while participants undertook a choice task designed to elicit preferences for a product. The task required participants to choose their preferred crackers described by shape (square, triangle, round), flavor (wheat, dark rye, plain) and topping (salt, poppy, no topping). The two main research objectives were (1) to observe and evaluate the cortical activity of the different brain regions and the interdependencies among the Electroencephalogram (EEG) signals from these regions; and (2) unlike most research in this area that has focused mainly on liking/disliking certain products, we provide a way to quantify the importance of different cracker features that contribute to the product design based on mutual information. We used the commercial Emotiv EPOC wireless EEG headset with 14 channels to collect EEG signals from participants. We also used a Tobii-Studio eye tracker system to relate the EEG data to the specific choice options (crackers). Subjects were shown 57 choice sets; each choice set described three choice options (crackers). The patterns of cortical activity were obtained in the five principal frequency bands, Delta (0–4 Hz), Theta (3–7 Hz), Alpha (8–12 Hz), Beta (13–30 Hz), and Gamma (30–40 Hz). There was a clear phase synchronization between the left and right frontal and occipital regions indicating interhemispheric communications during the chosen task for the 18 participants. Results also indicated that there was a clear and significant change (p < 0.01) in the EEG power spectral activities taking a place mainly in the frontal (delta, alpha and beta across F3, F4, FC5 and FC6), temporal (alpha, beta, gamma across T7), and occipital (theta, alpha, and beta across O1) regions when participants indicated their preferences for their preferred crackers. Additionally, our mutual information analysis indicated that the various cracker flavors and toppings of the crackers were more important factors affecting the buying decision than the shapes of the crackers.     

Action recognition via bio-inspired features: The richness of center–surround interaction

Motion is a key feature for a wide class of computer vision approaches to recognize actions. In this article, we show how to define bio-inspired features for action recognition. To do so, we start from a well-established bio-inspired motion model of cortical areas V1 and MT. The primary visual cortex, designated as V1, is the first cortical area encountered in the visual stream processing and early responses of V1 cells consist in tiled sets of selective spatiotemporal filters. The second cortical area of interest in this article is area MT where MT cells pool incoming information from V1 according to the shape and characteristic of their receptive field. To go beyond the classical models and following the observations from Xiao et al. [61], we propose here to model different surround geometries for MT cells receptive fields. Then, we define the so-called bio-inspired features associated to an input video, based on the average activity of MT cells. Finally, we show how these features can be used in a standard classification method to perform action recognition. Results are given for the Weizmann and KTH databases. Interestingly, we show that the diversity of motion representation at the MT level (different surround geometries), is a major advantage for action recognition. On the Weizmann database, the inclusion of different MT surround geometries improved the recognition rate from 63.01 ± 2.07% up to 99.26 ± 1.66% in the best case. Similarly, on the KTH database, the recognition rate was significantly improved with the inclusion of MT different surround geometries (from 47.82 ± 2.71% up to 92.44 ± 0.01% in the best case). We also discussed the limitations of the current approach which are closely related to the input video duration. These promising results encourage us to further develop bio-inspired models incorporating other brain mechanisms and cortical areas in order to deal with more complex videos.