A fully complex-valued radial basis function classifier for real-valued classification problems

In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier.     

Thalassaemia classification by neural networks and genetic programming

This paper presents the use of a neural network and a decision tree, which is evolved by genetic programming (GP), in thalassaemia classification. The aim is to differentiate between thalassaemic patients, persons with thalassaemia trait and normal subjects by inspecting characteristics of red blood cells, reticulocytes and platelets. A structured representation on genetic algorithms for non-linear function fitting or STROGANOFF is the chosen architecture for genetic programming implementation. For comparison, multilayer perceptrons are explored in classification via a neural network. The classification results indicate that the performance of the GP-based decision tree is approximately equal to that of the multilayer perceptron with one hidden layer. But the multilayer perceptron with two hidden layers, which is proven to have the most suitable architecture among networks with different number of hidden layers, outperforms the GP-based decision tree. Nonetheless, the structure of the decision tree reveals that some input features have no effects on the classification performance. The results confirm that the classification accuracy of the multilayer perceptron with two hidden layers can still be maintained after the removal of the redundant input features. Detailed analysis of the classification errors of the multilayer perceptron with two hidden layers, in which a reduced feature set is used as the network input, is also included. The analysis reveals that the classification ambiguity and misclassification among persons with minor thalassaemia trait and normal subjects is the main cause of classification errors. These results suggest that a combination of a multilayer perceptron with a blood cell analysis may give rise to a guideline/hint for further investigation of thalassaemia classification.     

Orthogonality of decision boundaries in complex-valued neural networks

This letter presents some results of an analysis on the decision boundaries of complex-valued neural networks whose weights, threshold values, input and output signals are all complex numbers. The main results may be summarized as follows. (1) A decision boundary of a single complex-valued neuron consists of two hypersurfaces that intersect orthogonally, and divides a decision region into four equal sections. The XOR problem and the detection of symmetry problem that cannot be solved with two-layered real-valued neural networks, can be solved by two-layered complex-valued neural networks with the orthogonal decision boundaries, which reveals a potent computational power of complex-valued neural nets. Furthermore, the fading equalization problem can be successfully solved by the two-layered complex-valued neural network with the highest generalization ability. (2) A decision boundary of a three-layered complex-valued neural network has the orthogonal property as a basic structure, and its two hypersurfaces approach orthogonality as all the net inputs to each hidden neuron grow. In particular, most of the decision boundaries in the three-layered complex-valued neural network inetersect orthogonally when the network is trained using Complex-BP algorithm. As a result, the orthogonality of the decision boundaries improves its generalization ability. (3) The average of the learning speed of the Complex-BP is several times faster than that of the Real-BP. The standard deviation of the learning speed of the Complex-BP is smaller than that of the Real-BP.It seems that the complex-valued neural network and the related algorithm are natural for learning complex-valued patterns for the above reasons.     

A connectionist approach to generating oblique decision trees

Neural networks and decision tree methods are two common approaches to pattern classification. While neural networks can achieve high predictive accuracy rates, the decision boundaries they form are highly nonlinear and generally difficult to comprehend. Decision trees, on the other hand, can be readily translated into a set of rules. In this paper, we present a novel algorithm for generating oblique decision trees that capitalizes on the strength of both approaches. Oblique decision trees classify the patterns by testing on linear combinations of the input attributes. As a result, an oblique decision tree is usually much smaller than the univariate tree generated for the same domain. Our algorithm consists of two components: connectionist and symbolic. A three-layer feedforward neural network is constructed and pruned, a decision tree is then built from the hidden unit activation values of the pruned network. An oblique decision tree is obtained by expressing the activation values using the original input attributes. We test our algorithm on a wide range of problems. The oblique decision trees generated by the algorithm preserve the high accuracy of the neural networks, while keeping the explicitness of decision trees. Moreover, they outperform univariate decision trees generated by the symbolic approach and oblique decision trees built by other approaches in accuracy and tree size.     

Sensitive deep convolutional neural network for face recognition at large standoffs with small dataset

In this paper, we propose a sensitive convolutional neural network which incorporates sensitivity term in the cost function of Convolutional Neural Network (CNN) to emphasize on the slight variations and high frequency components in highly blurred input image samples. The proposed cost function in CNN has a sensitivity part in which the conventional error is divided by the derivative of the activation function, and subsequently the total error is minimized by the gradient descent method during the learning process. Due to the proposed sensitivity term, the data samples at the decision boundaries appear more on the middle band or the high gradient part of the activation function. This highlights the slight changes in the highly blurred input images enabling better feature extraction resulting in better generalization and improved classification performance in the highly blurred images. To study the effect of the proposed sensitivity term, experiments were performed for the face recognition task on small dataset of facial images at different long standoffs in both night-time and day-time modalities.     

A new pruning heuristic based on variance analysis of sensitivityinformation

Architecture selection is a very important aspect in the design of neural networks (NNs) to optimally tune performance and computational complexity. Sensitivity analysis has been used successfully to prune irrelevant parameters from feedforward NNs. This paper presents a new pruning algorithm that uses the sensitivity analysis to quantify the relevance of input and hidden units. A new statistical pruning heuristic is proposed, based on the variance analysis, to decide which units to prune. The basic idea is that a parameter with a variance in sensitivity not significantly different from zero, is irrelevant and can be removed. Experimental results show that the new pruning algorithm correctly prunes irrelevant input and hidden units. The new pruning algorithm is also compared with standard pruning algorithms.     

Fast learning Circular Complex-valued Extreme Learning Machine (CC-ELM) for real-valued classification problems

In this paper, we present a fast learning fully complex-valued extreme learning machine classifier, referred to as ‘Circular Complex-valued Extreme Learning Machine (CC-ELM)’ for handling real-valued classification problems. CC-ELM is a single hidden layer network with non-linear input and hidden layers and a linear output layer. A circular transformation with a translational/rotational bias term that performs a one-to-one transformation of real-valued features to the complex plane is used as an activation function for the input neurons. The neurons in the hidden layer employ a fully complex-valued Gaussian-like (‘sech’) activation function. The input parameters of CC-ELM are chosen randomly and the output weights are computed analytically. This paper also presents an analytical proof to show that the decision boundaries of a single complex-valued neuron at the hidden and output layers of CC-ELM consist of two hyper-surfaces that intersect orthogonally. These orthogonal boundaries and the input circular transformation help CC-ELM to perform real-valued classification tasks efficiently.Performance of CC-ELM is evaluated using a set of benchmark real-valued classification problems from the University of California, Irvine machine learning repository. Finally, the performance of CC-ELM is compared with existing methods on two practical problems, viz., the acoustic emission signal classification problem and a mammogram classification problem. These study results show that CC-ELM performs better than other existing (both) real-valued and complex-valued classifiers, especially when the data sets are highly unbalanced.     

On-line system identification of complex systems using Chebyshev neural networks

This paper proposes a computationally efficient artificial neural network (ANN) model for system identification of unknown dynamic nonlinear discrete time systems. A single layer functional link ANN is used for the model where the need of hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials. Thus, creation of nonlinear decision boundaries in the multidimensional input space and approximation of complex nonlinear systems becomes easier. These models are linear in their parameters and nonlinear in the inputs. The recursive least squares method with forgetting factor is used as on-line learning algorithm for parameter updation. The good behaviour of the identification method is tested on Box and Jenkins Gas furnace benchmark identification problem, single input single output (SISO) and multi input multi output (MIMO) discrete time plants. Stability of the identification scheme is also addressed.     

Multiple kernel learning using single stage function approximation for binary classification problems

In this paper, the multiple kernel learning (MKL) is formulated as a supervised classification problem. We dealt with binary classification data and hence the data modelling problem involves the computation of two decision boundaries of which one related with that of kernel learning and the other with that of input data. In our approach, they are found with the aid of a single cost function by constructing a global reproducing kernel Hilbert space (RKHS) as the direct sum of the RKHSs corresponding to the decision boundaries of kernel learning and input data and searching that function from the global RKHS, which can be represented as the direct sum of the decision boundaries under consideration. In our experimental analysis, the proposed model had shown superior performance in comparison with that of existing two stage function approximation formulation of MKL, where the decision functions of kernel learning and input data are found separately using two different cost functions. This is due to the fact that single stage representation helps the knowledge transfer between the computation procedures for finding the decision boundaries of kernel learning and input data, which inturn boosts the generalisation capacity of the model.     

催化裂化过程稳态优化控制研究

催化裂化装置是一个高度非线性、时变、长时延、强耦合、分布参数和不确定性的复杂 系统.在研究其过程机理的基础上,定义了一种模糊神经网络用以建模,用自相关函数检验法检 验模型的正确性,再用改进的Frank-Wolfe算法进行稳态优化计算,并以一炼油厂催化裂化装 置为对象进行试验,研究其辨识、建模和稳态优化控制.这种模糊神经网络具有隐层数多、隐层 结点数多、泛化能力和逼近能力强、收敛速度快的优点,更突出的特点还在于可由输出端对输入 求导,为稳态优化计算提供了极大方便,它与改进的Frank-Wolfe算法相结合用于解决非线性 复杂生产过程的建模和稳态优化控制问题是可行的.     

Fuzzy rough granular neural networks, fuzzy granules, and classification

We introduce a fuzzy rough granular neural network (FRGNN) model based on the multilayer perceptron using a back-propagation algorithm for the fuzzy classification of patterns. We provide the development strategy of the network mainly based upon the input vector, initial connection weights determined by fuzzy rough set theoretic concepts, and the target vector. While the input vector is described in terms of fuzzy granules, the target vector is defined in terms of fuzzy class membership values and zeros. Crude domain knowledge about the initial data is represented in the form of a decision table, which is divided into subtables corresponding to different classes. The data in each decision table is converted into granular form. The syntax of these decision tables automatically determines the appropriate number of hidden nodes, while the dependency factors from all the decision tables are used as initial weights. The dependency factor of each attribute and the average degree of the dependency factor of all the attributes with respect to decision classes are considered as initial connection weights between the nodes of the input layer and the hidden layer, and the hidden layer and the output layer, respectively. The effectiveness of the proposed FRGNN is demonstrated on several real-life data sets.     

字词融合的双通道混合神经网络情感分析模型

针对双向门控循环神经网络(BiGRU)无法获取文本局部特征,卷积神经网络(CNN)无法聚焦文本全局特征的问题,提出一种字词融合的双通道混合神经网络文本情感分析模型(CW_BGCA).首先,将文本分别用字符级词向量和词语级词向量表示;然后使用门控循环神经网络和卷积神经网络结合的混合神经模型分别从字向量和词向量中提取隐层特...     

FRBF: A Fuzzy Radial Basis Function Network

The FRBF network is designed by integrating the principles of a radial basis function network and the fuzzy c-means algorithm. The architecture of the network is suitably modified at the hidden layer to realise a novel neural implementation of the fuzzy clustering algorithm. Fuzzy set-theoretic concepts are incorporated at the input, output and hidden layers, enabling the model to handle both linguistic and numeric inputs, and providing a soft output decision. The effectiveness of the model is demonstrated on a speech recognition problem.     

Median radial basis function neural network

Radial basis functions (RBFs) consist of a two-layer neural network, where each hidden unit implements a kernel function. Each kernel is associated with an activation region from the input space and its output is fed to an output unit. In order to find the parameters of a neural network which embeds this structure we take into consideration two different statistical approaches. The first approach uses classical estimation in the learning stage and it is based on the learning vector quantization algorithm and its second-order statistics extension. After the presentation of this approach, we introduce the median radial basis function (MRBF) algorithm based on robust estimation of the hidden unit parameters. The proposed algorithm employs the marginal median for kernel location estimation and the median of the absolute deviations for the scale parameter estimation. A histogram-based fast implementation is provided for the MRBF algorithm. The theoretical performance of the two training algorithms is comparatively evaluated when estimating the network weights. The network is applied in pattern classification problems and in optical flow segmentation.     

Uncertainty Estimation Using Fuzzy Measures for Multiclass Classification

Uncertainty arises in classification problems when the input pattern is not perfect or measurement error is unavoidable. In many applications, it would be beneficial to obtain an estimate of the uncertainty associated with a new observation and its membership within a particular class. Although statistical classification techniques base decision boundaries according to the probability distributions of the patterns belonging to each class, they are poor at supplying uncertainty information for new observations. Previous research has documented a multiarchitecture, monotonic function neural network model for the representation of uncertainty associated with a new observation for two-class classification. This paper proposes a modification to the monotonic function model to estimate the uncertainty associated with a new observation for multiclass classification. The model, therefore, overcomes a limitation of traditional classifiers that base decisions on sharp classification boundaries. As such, it is believed that this method will have advantages for applications such as biometric recognition in which the estimation of classification uncertainty is an important issue. This approach is based on the transformation of the input pattern vector relative to each classification class. Separate, monotonic, single-output neural networks are then used to represent the "degree-of-similarity" between each input pattern vector and each class. An algorithm for the implementation of this approach is proposed and tested with publicly available face-recognition data sets. The results indicate that the suggested approach provides similar classification performance to conventional principle component analysis (PCA) and linear discriminant analysis (LDA) techniques for multiclass pattern recognition problems as well as providing uncertainty information caused by misclassification     

基于股票市场灵敏度分析的神经网络预测模型

股票市场是非线性系统,具有内部结构复杂性和外部因素多变性,建立基于股票市场灵敏度分析的神经网络预测模型。针对神经网络结构设计问题,计算网络输入层与隐层神经元的灵敏度,并修剪网络中不敏感的神经元,在保证模型泛化能力的同时,实现网络结构精简;针对神经网络黑箱问题,根据输入层神经元灵敏度解决各输入变量对股票市场的重要性和反馈机制。以上证指数为例,在不同的时间跨度下对股票市场运行规律进行学习,并分析不同结构修剪模型的适用性和市场意义。最后,通过与其他神经网络预测模型比较,验证本文模型的有效性。     

Neural network as a decision support system in the development of pharmaceutical formulation—focus on solid dispersions

The aim of the study was to create simple, neural model of ketoprofen (Ket) dissolution from solid dispersions (SD) and physical mixtures (PM), which could be an aid in prospective development of pharmaceutical formulation. An application of artificial neural networks (ANNs) methodology was investigated using experimental data. Backpropagation (BP) ANNs with two hidden layers, hyperbolic tangent as the activation function and Hampel's target function were studied. Neuro-fuzzy systems were also applied. As the input variables formulation type and preparation technology as well as qualitative and quantitative composition of SD and physical mixtures (PM) were selected. Direct incorporation of physicochemical properties of excipients (connectivity index, CI) enhanced ANNs model usability. Further improvement of neural model was achieved by input variables reduction performed on the basis of the sensitivity analysis. ANNs functions as decision support system in prospective ketoprofen SD formulation as well as data-mining tool were confirmed.     

Road Safety Performance Function Analysis With Visual Feature Importance of Deep Neural Nets

Road safety performance function (SPF) analysis using data-driven and nonparametric methods, especially recent developed deep learning approaches, has gained increasing achievements. However, due to the learning mechanisms are hidden in a “black box” in deep learning, traffic features extraction and intelligent importance analysis are still unsolved and hard to generate. This paper focuses on this problem using a deciphered version of deep neural networks (DNN), one of the most popular deep learning models. This approach builds on visualization, feature importance and sensitivity analysis, can evaluate the contributions of input variables on model’s “black box” feature learning process and output decision. Firstly, a visual feature importance (ViFI) method that describes the importance of input features is proposed by adopting diagram and numerical-analysis. Secondly, by observing the change of weights using ViFI on unsupervised training and fine-tuning of DNN, the final contributions of input features are calculated according to importance equations for both steps that we proposed. Sequentially, a case study based on a road SPF analysis is demonstrated, using data collected from a major Canadian highway, Highway 401. The proposed method allows effective deciphering of the model’s inner workings and allows the significant features to be identified and the bad features to be eliminated. Finally, the revised dataset is used in crash modeling and vehicle collision prediction, and the testing result verifies that the deciphered and revised model achieves state-of-the-art performance.      

基于RS-CSA-ELM的WSN节点故障诊断

为及时发现WSN节点故障隐患,准确掌握WSN运行状态,本文利用粗糙集理论属性约简算法(简称RS)对WSN节点故障属性进行约简,以最优的故障属性决策表重构训练样本数据集,作为极限学习机(Extreme Learning Machine, ELM)神经网络的输入,建立一个数据驱动的WSN节点故障断模型.采用乌鸦搜索算法(Crow Search Algorithm, CSA)优化ELM神经网络的输入权值和隐含层阀值,改善网络参数随机生成带来的ELM模型输出不稳定、分类精度偏低的问题.通过对RS-GA-ELM模型进行仿真分析.结果表明, RS-GA-ELM模型能够在可靠性不同的数据集中,保持较高的故障诊断效率,符合WSN节点故障诊断的需求.     

Shallow and deep learning for image classification

The paper is focused on the idea to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying from such popular benchmark databases as FERET and MNIST. An autoassociative neural network is used as a standalone program realized the nonlinear principal component analysis for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. A special study of the optimal choice of activation function and the normalization transformation of input data allows to improve efficiency of the autoassociative program. One more study devoted to denoising properties of this program demonstrates its high efficiency even on noisy data. Three types of neural networks are compared: feed-forward neural net with one hidden layer, deep network with several hidden layers and deep belief network with several pretraining layers realized restricted Boltzmann machine. The number of hidden layer and the number of hidden neurons in them were chosen by cross-validation procedure to keep balance between number of layers and hidden neurons and classification efficiency. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both multiprocessor graphic card and cloud services to speed up our calculations. The paper is oriented to specialists in concrete fields of scientific or experimental applications, who have already some knowledge about artificial neural networks, probability theory and numerical methods.