Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

Fuzzy neural networks for intelligent design retrieval using associative manufacturing features

In the conceptual design stage, designers usually initiate a design concept through an association activity. The activity helps designers collect and retrieve reference information regarding a current design subject instead of starting from scratch. By modifying previous designs, designers can create a new design in a much shorter time. To computerize this process, this paper proposes an intelligent design retrieval system involving soft computing techniques for both feature and object association functions. A feature association method that utilizes fuzzy relation and fuzzy composition is developed to increase the searching spectrum. In the mean time, object association functions composed by a fuzzy neural network allow designers to control the similarity of retrieved designs. Our implementation result shows that the intelligent design retrieval system with two soft computing based association functions can retrieve target reference designs as expected.     

Uncertainty-aware soft sensor using Bayesian recurrent neural networks

Data-driven soft sensors have been widely used to measure key variables for industrial processes. Soft sensors using deep learning models have attracted considerable attention and shown superior predictive performance. However, if a soft sensor encounters an unexpected situation in inferring data or if noisy input data is used, the estimated value derived by a standard soft sensor using deep learning may at best be untrustworthy. This problem can be mitigated by expressing a degree of uncertainty about the trustworthiness of the estimated value produced by the soft sensor. To address this issue of uncertainty, we propose using an uncertainty-aware soft sensor that uses Bayesian recurrent neural networks (RNNs). The proposed soft sensor uses a RNN model as a backbone and is then trained using Bayesian techniques. The experimental results demonstrated that such an uncertainty-aware soft sensor increases the reliability of predictive uncertainty. In comparisons with a standard soft sensor, it shows a capability to use uncertainties for interval prediction without compromising predictive performance.     

Structural synthesis of fast two-layer neural networks

Methods of construction of structural models of fast two-layer neural networks are considered. The methods are based on the criteria of minimum computing operations and maximum degrees of freedom. Optimal structural models of two-layer neural networks are constructed. Illustrative examples are given. Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 47–56, July–August, 2000.     

Negative effects of sufficiently small initial weights on back-propagation neural networks

In the training of feedforward neural networks,it is usually suggested that the initial weights should be small in magnitude in order to prevent premature saturation.The aim of this paper is to point out the other side of the story:In some cases,the gradient of the error functions is zero not only for infinitely large weights but also for zero weights.Slow convergence in the beginning of the training procedure is often the result of sufficiently small initial weights.Therefore,we suggest that,in these cases,the initial values of the weights should be neither too large,nor too small.For instance,a typical range of choices of the initial weights might be something like(0.4,0.1) ∪(0.1,0.4),rather than(0.1,0.1) as suggested by the usual strategy.Our theory that medium size weights should be used has also been extended to a few commonly used transfer functions and error functions.Numerical experiments are carried out to support our theoretical findings.     

Cloud-based video analytics using convolutional neural networks

Object classification is a vital part of any video analytics system, which could aid in complex applications such as object monitoring and management. Traditional video analytics systems work on shallow networks and are unable to harness the power of distributed processing for training and inference. We propose a cloud-based video analytics system based on an optimally tuned convolutional neural network to classify objects from video streams. The tuning of convolutional neural network is empowered by in-memory distributed computing. The object classification is performed by comparing the target object with the prestored trained patterns, generating a set of matching scores. The matching scores greater than an empirically determined threshold reveal the classification of the target object. The proposed system proved to be robust to classification errors with an accuracy and precision of 97% and 96%, respectively, and can be used as a general-purpose video analytics system.     

VLSI implementation of pulsating neural networks

We describe a generic approach for realizing networks of pulsating neurons based on charge pumping of interface states situated in the channel of MOS transistors. Two basic building blocks will be described: the pulse activated charge pumping (PSCP) synapse, and the charge sensitive oscillator (CSO). The PSCP synapse which operates as either a short or a long term memory device which produces a charge packet proportional to the number of pulses applied to its input, will be described in detail together with experimental results demonstrating its capability. The CSO circuit which is a charge controlled oscillator will be described together with simulations of its output frequency dependence on its input voltage, and the relation between the temporal dependence of output waveform on its input charge.     

Review of pulse-coupled neural networks

This paper reviews the research status of pulse-coupled neural networks (PCNN) in the past decade. Considering there are too many publications about the PCNN, we summarize main approaches and point out interesting parts of the PCNN researches rather than contemplate to go into details of particular algorithms or describe results of comparative experiments. First, the current status of the PCNN and some modified models are briefly introduced. Second, we review the PCNN applications in the field of image processing (e.g. image segmentation, image enhancement, image fusion, object and edge detection, pattern recognition, etc.), then applications in other fields also are mentioned. Subsequently, some existing problems are summarized, while we give some suggestions for the solutions to some puzzles. Finally, the trend of the PCNN is pointed out.     

Modifying genetic programming for artificial neural network development for data mining

The development of artificial neural networks (ANNs) is usually a slow process in which the human expert has to test several architectures until he finds the one that achieves best results to solve a certain problem. However, there are some tools that provide the ability of automatically developing ANNs, many of them using evolutionary computation (EC) tools. One of the main problems of these techniques is that ANNs have a very complex structure, which makes them very difficult to be represented and developed by these tools. This work presents a new technique that modifies genetic programming (GP) so as to correctly and efficiently work with graph structures in order to develop ANNs. This technique also allows the obtaining of simplified networks that solve the problem with a small group of neurons. In order to measure the performance of the system and to compare the results with other ANN development methods by means of evolutionary computation (EC) techniques, several tests were performed with problems based on some of the most used test databases in the Data Mining domain. These comparisons show that the system achieves good results that are not only comparable to those of the already existing techniques but, in most cases, improve them.     

Classification error of multilayer perceptron neural networks

In subject classification, artificial neural networks (ANNS) are efficient and objective classification methods. Thus, they have been successfully applied to the numerous classification fields. Sometimes, however, classifications do not match the real world, and are subjected to errors. These problems are caused by the nature of ANNS. We discuss these on multilayer perceptron neural networks. By studying of these problems, it helps us to have a better understanding on its classification.     

Statistical fuzzy interval neural networks for currency exchange rate time series prediction

In this paper, the statistical fuzzy interval neural network with statistical interval input and output values is proposed to perform statistical fuzzy knowledge discovery and the currency exchange rate prediction. Time series data sets are grouped into time series data granules with statistical intervals. The statistical interval data sets including week-based averages, maximum errors of estimate and standard deviations are used to train the fuzzy interval neural network to discover fuzzy IF-THEN rules. The output of the fuzzy interval neural network is an interval value with certain percent confidence. Simulations are completed in terms of the exchange rates between US Dollar and other three currencies (Japanese Yen, British Pound and Hong Kong Dollar). The simulation results show that the fuzzy interval neural network can provide more tolerant prediction results.     

Nonsmooth training of fuzzy neural networks

The integration of fuzzy methods and neural networks often leads to nonsmoothness of the neural network and, consequently, to a nonsmooth training problem. It is shown, that smooth training methods as e.g. backpropagation fail to converge in this case. Thus a method – based on so called bundle-methods – for training of nonsmooth neural network is presented. Numerical results obtained from a character recognition problem show, that this method still converges where backpropagation fails.     

GA-based learning for rule identification in fuzzy neural networks

Employing an effective learning process is a critical topic in designing a fuzzy neural network, especially when expert knowledge is not available. This paper presents a genetic algorithm (GA) based learning approach for a specific type of fuzzy neural network. The proposed learning approach consists of three stages. In the first stage the membership functions of both input and output variables are initialized by determining their centers and widths using a self-organizing algorithm. The second stage employs the proposed GA based learning algorithm to identify the fuzzy rules while the final stage tunes the derived structure and parameters using a back-propagation learning algorithm. The capabilities of the proposed GA-based learning approach are evaluated using a well-examined benchmark example and its effectiveness is analyzed by means of a comparative study with other approaches. The usefulness of the proposed GA-based learning approach is also illustrated in a practical case study where it is used to predict the performance of road traffic control actions. Results from the benchmarking exercise and case study effectively demonstrate the ability of the proposed three stages learning approach to identify relevant fuzzy rules from a training data set with a higher prediction accuracy than alternative approaches.     

Evolutionary neural networks and DNA computing algorithms for dual-axis motion control

A new method is proposed to deal with the dual-axis control of a multi-variables system with two induction motors. Investigation of resolving the cross-coupling problem of dual-axis platform is addressed by a neural net-based decoupling compensator and a sufficient condition ensuring closed-loop stability is derived. An evolutionary algorithm processing the universal seeking capability is proposed for finding the optimal connecting weights of the neural decoupling compensator and the gains of PID controllers. Extensive numerical studies verify the performance and applicability of the proposed design under a variety of operating conditions.     

Parallelization of cellular neural networks on GPU

Recently, cellular neural networks (CNNs) have been demonstrated to be a highly effective paradigm applicable in a wide range of areas. Typically, CNNs can be implemented using VLSI circuits, but this would unavoidably require additional hardware. On the other hand, we can also implement CNNs purely by software; this, however, would result in very low performance when given a large CNN problem size. Nowadays, conventional desktop computers are usually equipped with programmable graphics processing units (GPUs) that can support parallel data processing. This paper introduces a GPU-based CNN simulator. In detail, we carefully organize the CNN data as 4-channel textures, and efficiently implement the CNN computation as fragment programs running in parallel on a GPU. In this way, we can create a high performance but low-cost CNN simulator. Experimentally, we demonstrate that the resultant GPU-based CNN simulator can run 8–17 times faster than a CPU-based CNN simulator.     

Artificial neural networks: a review of commercial hardware

Artificial neural networks (ANN) became a common solution for a wide variety of problems in many fields, such as control and pattern recognition to name but a few. Many solutions found in these and other ANN fields have reached a hardware implementation phase, either commercial or with prototypes. The most frequent solution for the implementation of ANN consists of training and implementing the ANN within a computer. Nevertheless this solution might be unsuitable because of its cost or its limited speed. The implementation might be too expensive because of the computer and too slow when implemented in software. In both cases dedicated hardware can be an interesting solution.

The necessity of dedicated hardware might not imply building the hardware since in the last two decades several commercial hardware solutions that can be used in the implementation have reached the market.

Unfortunately not every integrated circuit will fit the needs: some will use lower precision, some will implement only certain types of networks, some don’t have training built in and the information is not easy to find.

This article is confined to reporting the commercial chips that have been developed specifically for ANN, leaving out other solutions.

This option has been made because most of the other solutions are based on cards which are built either with these chips, Digital Signal Processors or Reduced Instruction Set Computers.     

基于联合卷积和递归神经网络的交通标志识别

提出了一种联合卷积和递归神经网络的深层网络结构,在卷积神经网络中引入了递归神经网络能学到的组合特征:原始图片先通过一级由k均值聚类学得滤波器的卷积神经网络,得到的结果再同时通过一级卷积和一级递归神经网络,最后得到的特征向量由Softmax分类器进行分类。实验结果表明:在第二级卷积和递归神经网络权重随机的情况下,该网络的识别率已经能够达到98.28%,跟其他网络结构相比,大大减少了训练时间,而且无需复杂的工程技巧。     

基于网格的神经网络计算平台的架构

介绍了神经网络计算平台在网格上的架构。由于神经网络应用环境的复杂性及要求处理大量数据的问题,从而神经网络计算需要超强的计算能力,因此在神经网络计算平台中引入了网格的思想,力图建立一个基于网格的神经网络计算平台。平台使用Globus工具,采用了统一控制和完全托管的思想。鉴于神经网络算法的复杂性和大数据量,神经计算算法在网格上进行适合各自特性的分解。     

Predicting stock index increments by neural networks: The role of trading volume under different horizons

Recent studies show that there is a significant bidirectional nonlinear causality between stock return and trading volume. In this research, we reinforce this statement and the results presented in some earlier literatures and further investigate whether trading volume can significantly improve the prediction performance of neural networks under short-, medium-and long-term forecasting horizons. An application of component-based neural networks is used in forecasting one-step ahead stock index increments. The models are also augmented by the addition of different combinations of indices’ and component stocks’ trading volumes as inputs to form more general ex-ante forecasting models. Neural networks are trained with the data of stock returns and volumes from NASDAQ, DJIA and STI indices. Results indicate that augmented neural network models with trading volumes lead to improvements, at different extents, in forecasting performance under different terms of forecasting horizon. Empirical results indicate that trading volumes lead to modest improvements on the performance of stock index increments prediction under medium-and long-term horizons.