Boosting neural network feature extraction by reduced accuracy activation functions

The paper proposes the use of a lookup table in the computation of the sigmoid function in multilayer perceptron feature extraction. The approach is frequently used in limited precision hardware implementations. This paper considers its use as a software implementation. It is argued that the method is computationally efficient and does not compromise the performance of the neural networks. This new approach is analysed and evaluated in both theoretical and experimental terms. The practical study is performed using several real world and synthetic data sets. The results show that an increase in computational efficiency is achieved with very little or no influence on the quality of the extracted features.     

Classification trees with neural network feature extraction

The ideal use of small multilayer nets at the decision nodes of a binary classification tree to extract nonlinear features is proposed. The nets are trained and the tree is grown using a gradient-type learning algorithm in the multiclass case. The method improves on standard classification tree design methods in that it generally produces trees with lower error rates and fewer nodes. It also reduces the problems associated with training large unstructured nets and transfers the problem of selecting the size of the net to the simpler problem of finding a tree of the right size. An efficient tree pruning algorithm is proposed for this purpose. Trees constructed with the method and the CART method are compared on a waveform recognition problem and a handwritten character recognition problem. The approach demonstrates significant decrease in error rate and tree size. It also yields comparable error rates and shorter training times than a large multilayer net trained with backpropagation on the same problems.     

Transformation radial basis neural network for relevant feature selection

Radial basis neural networks are excellent candidates for selecting relevant features in pattern recognition problems. By a slight change in the traditional three-layer architecture of a radial basis neural network, we can obtain a quantitative method, which allows us to get a ranking within the features. We present a new neural network concept, combining at the same time two different skills: classification and detection of relevant features in the input vector.     

Unsupervised neural network learning procedures for feature extraction and classification

In this article, we review unsupervised neural network learning procedures which can be applied to the task of preprocessing raw data to extract useful features for subsequent classification. The learning algorithms reviewed here are grouped into three sections: information-preserving methods, density estimation methods, and feature extraction methods. Each of these major sections concludes with a discussion of successful applications of the methods to real-world problems.The first author is supported by research grants from the James S. McDonnell Foundation (grant #93–95) and the Natural Sciences and Engineering Research Council of Canada. For part of this work, the second author was supported by a Temporary Lectureship from the Academic Initiative of the University of London, and by a grant (GR/J38987) from the Science and Engineering Research Council (SERC) of the UK.     

A hybrid neural network/genetic algorithm approach to optimizing feature extraction for signal classification.

In this paper, a hybrid neural network/genetic algorithm technique is presented, aiming at designing a feature extractor that leads to highly separable classes in the feature space. The application upon which the system is built, is the identification of the state of human peripheral vascular tissue (i.e., normal, fibrous and calcified). The system is further tested on the classification of spectra measured from the cell nucleii in blood samples in order to distinguish normal cells from those affected by Acute Lymphoblastic Leukemia. As advantages of the proposed technique we may encounter the algorithmic nature of the design procedure, the optimized classification results and the fact that the system performance is less dependent on the classifier type to be used.     

基于角微特征的神经网络识别人脸特征

提出一种利用人脸角微特征几何特性的图像预处理,建立BP神经网络识别人脸特征模型的方法。研究了角微特征提取和具体算法,讨论了BP网络结构的设计,输入、输出层设计和隐藏层节点选取问题。微特征提取,可以降低网络输入维度,对于识别不同角度、不同表情的人脸图像提供了可能性。利用ORL人脸图像数据库做实验,结果表明此方法有效。     

基于余弦基神经网络求震荡函数积分方法

针对震荡函数数值积分计算问题,提出了一种基于余弦基函数神经网络模型和学习算法,将该算法应用于求解震荡函数数值积分.通过算例,计算机仿真实验表明,提出的算法相比传统的震荡函数数值积分方法,具有模型简单、计算精度较高、收敛速度快等特点.     

A new genetic feature selection with neural network ensemble

A neural network ensemble is a learning paradigm in which a finite collection of neural networks is trained for the same task. Ensembles generally show better classification and generalization performance than a single neural network does. In this paper, a new feature selection method for a neural network ensemble is proposed for pattern classification. The proposed method selects an adequate feature subset for each constituent neural network of the ensemble using a genetic algorithm. Unlike the conventional feature selection method, each neural network is only allowed to have some (not all) of the considered features. The proposed method can therefore be applied to huge-scale feature classification problems. Experiments are performed with four databases to illustrate the performance of the proposed method.     

A neural network model for monotone linear asymmetric variationalinequalities

A linear variational inequality is a uniform approach for some important problems in optimization and equilibrium problems. We give a neural network model for solving asymmetric linear variational inequalities. The model is based on a simple projection and contraction method. Computer simulation is performed for linear programming (LP) and linear complementarity problems (LCP). The test results for the LP problem demonstrate that our model converges significantly faster than the three existing neural network models examined in a comparative study paper.     

Discrete derivative approximations with scale-space properties: A basis for low-level feature extraction

This article shows how discrete derivative approximations can be defined so thatscale-space properties hold exactly also in the discrete domain. Starting from a set of natural requirements on the first processing stages of a visual system,the visual front end, it gives an axiomatic derivation of how a multiscale representation of derivative approximations can be constructed from a discrete signal, so that it possesses analgebraic structure similar to that possessed by the derivatives of the traditional scale-space representation in the continuous domain. A family of kernels is derived that constitutediscrete analogues to the continuous Gaussian derivatives.The representation has theoretical advantages over other discretizations of the scale-space theory in the sense that operators that commute before discretizationcommute after discretization. Some computational implications of this are that derivative approximations can be computeddirectly from smoothed data and that this will giveexactly the same result as convolution with the corresponding derivative approximation kernel. Moreover, a number ofnormalization conditions are automatically satisfied.The proposed methodology leads to a scheme of computations of multiscale low-level feature extraction that is conceptually very simple and consists of four basic steps: (i)large support convolution smoothing, (ii)small support difference computations, (iii)point operations for computing differential geometric entities, and (iv)nearest-neighbour operations for feature detection.Applications demonstrate how the proposed scheme can be used for edge detection and junction detection based on derivatives up to order three.     

Decision boundary feature extraction for neural networks

In this paper, we propose a new feature extraction method for feedforward neural networks. The method is based on the recently published decision boundary feature extraction algorithm which is based on the fact that all the necessary features for classification can be extracted from the decision boundary. The decision boundary feature extraction algorithm can take advantage of characteristics of neural networks which can solve complex problems with arbitrary decision boundaries without assuming underlying probability distribution functions of the data. To apply the decision boundary feature extraction method, we first give a specific definition for the decision boundary in a neural network. Then, we propose a procedure for extracting all the necessary features for classification from the decision boundary. Experiments show promising results.     

基于动态特征提取和神经网络的数据流分类研究

为提高数据流分类的精确性和适应性,提出了一种新的数据流分类方法。该方法基于总体最小二乘法对数据流进行分段拟合,并将传统曲线分析算法——滑动窗口(SW)和在线数据分割(OSD)进行结合、改进,以可变滑动窗口算法实现对数据流的合理分割,提高趋势分析精度。在此基础上,对数据流进行动态特征提取和判断,并以神经网络对数据流特征进行模式识别,精确分类,进而对监控对象提供早期预警、状态评估和决策支持。实验结果表明,该方法能对数据流进行有效的动态特征描述,分类效果明显。     

Dictionary learning for VQ feature extraction in ECG beats classification

Vector quantization(VQ) can perform efficient feature extraction from electrocardiogram (ECG) with the advantages of dimensionality reduction and accuracy increase. However, the existing dictionary learning algorithms for vector quantization are sensitive to dirty data, which compromises the classification accuracy. To tackle the problem, we propose a novel dictionary learning algorithm that employs k-medoids cluster optimized by k-means++ and builds dictionaries by searching and using representative samples, which can avoid the interference of dirty data, and thus boost the classification performance of ECG systems based on vector quantization features. We apply our algorithm to vector quantization feature extraction for ECG beats classification, and compare it with popular features such as sampling point feature, fast Fourier transform feature, discrete wavelet transform feature, and with our previous beats vector quantization feature. The results show that the proposed method yields the highest accuracy and is capable of reducing the computational complexity of ECG beats classification system. The proposed dictionary learning algorithm provides more efficient encoding for ECG beats, and can improve ECG classification systems based on encoded feature.     

A 3D neural network for moving microorganism extraction

Neural Computing and Applications - Accurate detection and extraction of moving microorganisms from microscopic video streams is the first important step in biological wastewater treatment system....     

Deep learning neural network for texture feature extraction in oral cancer: enhanced loss function

Multimedia Tools and Applications - The use of a binary classifier like the sigmoid function and loss functions reduces the accuracy of deep learning algorithms. This research aims to increase the...     

基于特征提取和RBF神经网络的ECT流型辨识

针对传统ECT流型辨识方法效率低的问题,提出了一种基于特征提取和径向基函数神经网络相结合的ECT图像流型辨识的方法,该方法通过对各种特征参数的定义,完成对ECT系统测得的电容值进行特征提取,然后将提取的特征值作为RBF神经网络的输入完成流型辨识。仿真和实验结果表明,与基于BP神经网络的图像流型辨识方法相比,该方法具有识别速度快和效率高等优点,为ECT图像流型识别的研究提供了一个新的思路。     

A novel radial basis function neural network for discriminant analysis

A novel radial basis function neural network for discriminant analysis is presented in this paper. In contrast to many other researches, this work focuses on the exploitation of the weight structure of radial basis function neural networks using the Bayesian method. It is expected that the performance of a radial basis function neural network with a well-explored weight structure can be improved. As the weight structure of a radial basis function neural network is commonly unknown, the Bayesian method is, therefore, used in this paper to study this a priori structure. Two weight structures are investigated in this study, i.e., a single-Gaussian structure and a two-Gaussian structure. An expectation-maximization learning algorithm is used to estimate the weights. The simulation results showed that the proposed radial basis function neural network with a weight structure of two Gaussians outperformed the other algorithms.     

A clustering algorithm for radial basis function neural network initialization

In this paper, we propose an Output-Constricted Clustering (OCC) algorithm for Radial Basis Function Neural Network (RBFNN) initialization. OCC first roughly partitions the output based on the required precision and then refinedly clusters data based on the input complexity within each output partition. The main contribution of the proposed clustering algorithm is that we introduce the concept of separability, which is a criterion to judge the suitability of the number of sub-clusters in each output partition. As a result, OCC is able to determine the proper number of sub-clusters with appropriate locations within each output partition by considering both input and output information. The resulting clusters from OCC are used to initialize RBFNN, with proper number and initial locations of for hidden neurons. As a result, RBFNN starting it's learning from a good point, is able to achieve better approximation performance than existing clustering methods for RBFNN initialization. This better performance is illustrated by a number of examples.     

A new method for classification of ECG arrhythmias using neural network with adaptive activation function

In this study, new neural network models with adaptive activation function (NNAAF) were implemented to classify ECG arrhythmias. Our NNAAF models included three types named as NNAAF-1, NNAAF-2 and NNAAf-3. Activation functions with adjustable free parameters were used in hidden neurons of these models to improve classical MLP network. In addition, these three NNAAF models were compared with the MLP model implemented in similar conditions. Ten different types of ECG arrhythmias were selected from MIT–BIH ECG Arrhythmias Database to train NNAAFs and MLP models. Moreover, all models tested by the ECG signals of 92 patients (40 males and 52 females, average age is $39.75\pm 19.06$). The average accuracy rate of all models in the training processing was found as 99.92%. The average accuracy rate of the all models in the test phases was obtained as 98.19.     

带扰动的变频正弦混沌神经网络研究

为了研究变频正弦混沌神经网络(FCSCNN)的抗扰动能力,在该混沌神经元的内部状态中分别引入三角函数和小波函数扰动项,提出了带扰动的变频正弦混沌神经元模型。给出了该混沌神经元的倒分岔图及Lyapunov指数的时间演化图,分析了其动力学特性。利用该模型构建了新型暂态混沌神经网络,通过选择不同的扰动系数,将其应用于函数优化和组合优化问题上。仿真实验表明,在适当的扰动系数下,变频正弦混沌神经网络能够有效地解决函数优化和组合优化问题,体现了该模型具有较强的鲁棒性和抗扰动能力。