Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

Fuzzy neural networks for intelligent design retrieval using associative manufacturing features

In the conceptual design stage, designers usually initiate a design concept through an association activity. The activity helps designers collect and retrieve reference information regarding a current design subject instead of starting from scratch. By modifying previous designs, designers can create a new design in a much shorter time. To computerize this process, this paper proposes an intelligent design retrieval system involving soft computing techniques for both feature and object association functions. A feature association method that utilizes fuzzy relation and fuzzy composition is developed to increase the searching spectrum. In the mean time, object association functions composed by a fuzzy neural network allow designers to control the similarity of retrieved designs. Our implementation result shows that the intelligent design retrieval system with two soft computing based association functions can retrieve target reference designs as expected.     

Structural synthesis of fast two-layer neural networks

Methods of construction of structural models of fast two-layer neural networks are considered. The methods are based on the criteria of minimum computing operations and maximum degrees of freedom. Optimal structural models of two-layer neural networks are constructed. Illustrative examples are given. Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 47–56, July–August, 2000.     

Negative effects of sufficiently small initial weights on back-propagation neural networks

In the training of feedforward neural networks,it is usually suggested that the initial weights should be small in magnitude in order to prevent premature saturation.The aim of this paper is to point out the other side of the story:In some cases,the gradient of the error functions is zero not only for infinitely large weights but also for zero weights.Slow convergence in the beginning of the training procedure is often the result of sufficiently small initial weights.Therefore,we suggest that,in these cases,the initial values of the weights should be neither too large,nor too small.For instance,a typical range of choices of the initial weights might be something like(0.4,0.1) ∪(0.1,0.4),rather than(0.1,0.1) as suggested by the usual strategy.Our theory that medium size weights should be used has also been extended to a few commonly used transfer functions and error functions.Numerical experiments are carried out to support our theoretical findings.     

VLSI implementation of pulsating neural networks

We describe a generic approach for realizing networks of pulsating neurons based on charge pumping of interface states situated in the channel of MOS transistors. Two basic building blocks will be described: the pulse activated charge pumping (PSCP) synapse, and the charge sensitive oscillator (CSO). The PSCP synapse which operates as either a short or a long term memory device which produces a charge packet proportional to the number of pulses applied to its input, will be described in detail together with experimental results demonstrating its capability. The CSO circuit which is a charge controlled oscillator will be described together with simulations of its output frequency dependence on its input voltage, and the relation between the temporal dependence of output waveform on its input charge.     

Review of pulse-coupled neural networks

This paper reviews the research status of pulse-coupled neural networks (PCNN) in the past decade. Considering there are too many publications about the PCNN, we summarize main approaches and point out interesting parts of the PCNN researches rather than contemplate to go into details of particular algorithms or describe results of comparative experiments. First, the current status of the PCNN and some modified models are briefly introduced. Second, we review the PCNN applications in the field of image processing (e.g. image segmentation, image enhancement, image fusion, object and edge detection, pattern recognition, etc.), then applications in other fields also are mentioned. Subsequently, some existing problems are summarized, while we give some suggestions for the solutions to some puzzles. Finally, the trend of the PCNN is pointed out.     

Modifying genetic programming for artificial neural network development for data mining

The development of artificial neural networks (ANNs) is usually a slow process in which the human expert has to test several architectures until he finds the one that achieves best results to solve a certain problem. However, there are some tools that provide the ability of automatically developing ANNs, many of them using evolutionary computation (EC) tools. One of the main problems of these techniques is that ANNs have a very complex structure, which makes them very difficult to be represented and developed by these tools. This work presents a new technique that modifies genetic programming (GP) so as to correctly and efficiently work with graph structures in order to develop ANNs. This technique also allows the obtaining of simplified networks that solve the problem with a small group of neurons. In order to measure the performance of the system and to compare the results with other ANN development methods by means of evolutionary computation (EC) techniques, several tests were performed with problems based on some of the most used test databases in the Data Mining domain. These comparisons show that the system achieves good results that are not only comparable to those of the already existing techniques but, in most cases, improve them.     

Classification error of multilayer perceptron neural networks

In subject classification, artificial neural networks (ANNS) are efficient and objective classification methods. Thus, they have been successfully applied to the numerous classification fields. Sometimes, however, classifications do not match the real world, and are subjected to errors. These problems are caused by the nature of ANNS. We discuss these on multilayer perceptron neural networks. By studying of these problems, it helps us to have a better understanding on its classification.     

Nonsmooth training of fuzzy neural networks

The integration of fuzzy methods and neural networks often leads to nonsmoothness of the neural network and, consequently, to a nonsmooth training problem. It is shown, that smooth training methods as e.g. backpropagation fail to converge in this case. Thus a method – based on so called bundle-methods – for training of nonsmooth neural network is presented. Numerical results obtained from a character recognition problem show, that this method still converges where backpropagation fails.     

Parallelization of cellular neural networks on GPU

Recently, cellular neural networks (CNNs) have been demonstrated to be a highly effective paradigm applicable in a wide range of areas. Typically, CNNs can be implemented using VLSI circuits, but this would unavoidably require additional hardware. On the other hand, we can also implement CNNs purely by software; this, however, would result in very low performance when given a large CNN problem size. Nowadays, conventional desktop computers are usually equipped with programmable graphics processing units (GPUs) that can support parallel data processing. This paper introduces a GPU-based CNN simulator. In detail, we carefully organize the CNN data as 4-channel textures, and efficiently implement the CNN computation as fragment programs running in parallel on a GPU. In this way, we can create a high performance but low-cost CNN simulator. Experimentally, we demonstrate that the resultant GPU-based CNN simulator can run 8–17 times faster than a CPU-based CNN simulator.     

GA-based learning for rule identification in fuzzy neural networks

Employing an effective learning process is a critical topic in designing a fuzzy neural network, especially when expert knowledge is not available. This paper presents a genetic algorithm (GA) based learning approach for a specific type of fuzzy neural network. The proposed learning approach consists of three stages. In the first stage the membership functions of both input and output variables are initialized by determining their centers and widths using a self-organizing algorithm. The second stage employs the proposed GA based learning algorithm to identify the fuzzy rules while the final stage tunes the derived structure and parameters using a back-propagation learning algorithm. The capabilities of the proposed GA-based learning approach are evaluated using a well-examined benchmark example and its effectiveness is analyzed by means of a comparative study with other approaches. The usefulness of the proposed GA-based learning approach is also illustrated in a practical case study where it is used to predict the performance of road traffic control actions. Results from the benchmarking exercise and case study effectively demonstrate the ability of the proposed three stages learning approach to identify relevant fuzzy rules from a training data set with a higher prediction accuracy than alternative approaches.     

基于联合卷积和递归神经网络的交通标志识别

提出了一种联合卷积和递归神经网络的深层网络结构,在卷积神经网络中引入了递归神经网络能学到的组合特征:原始图片先通过一级由k均值聚类学得滤波器的卷积神经网络,得到的结果再同时通过一级卷积和一级递归神经网络,最后得到的特征向量由Softmax分类器进行分类。实验结果表明:在第二级卷积和递归神经网络权重随机的情况下,该网络的识别率已经能够达到98.28%,跟其他网络结构相比,大大减少了训练时间,而且无需复杂的工程技巧。     

Evolutionary neural networks and DNA computing algorithms for dual-axis motion control

A new method is proposed to deal with the dual-axis control of a multi-variables system with two induction motors. Investigation of resolving the cross-coupling problem of dual-axis platform is addressed by a neural net-based decoupling compensator and a sufficient condition ensuring closed-loop stability is derived. An evolutionary algorithm processing the universal seeking capability is proposed for finding the optimal connecting weights of the neural decoupling compensator and the gains of PID controllers. Extensive numerical studies verify the performance and applicability of the proposed design under a variety of operating conditions.     

Artificial neural networks: a review of commercial hardware

Artificial neural networks (ANN) became a common solution for a wide variety of problems in many fields, such as control and pattern recognition to name but a few. Many solutions found in these and other ANN fields have reached a hardware implementation phase, either commercial or with prototypes. The most frequent solution for the implementation of ANN consists of training and implementing the ANN within a computer. Nevertheless this solution might be unsuitable because of its cost or its limited speed. The implementation might be too expensive because of the computer and too slow when implemented in software. In both cases dedicated hardware can be an interesting solution.

The necessity of dedicated hardware might not imply building the hardware since in the last two decades several commercial hardware solutions that can be used in the implementation have reached the market.

Unfortunately not every integrated circuit will fit the needs: some will use lower precision, some will implement only certain types of networks, some don’t have training built in and the information is not easy to find.

This article is confined to reporting the commercial chips that have been developed specifically for ANN, leaving out other solutions.

This option has been made because most of the other solutions are based on cards which are built either with these chips, Digital Signal Processors or Reduced Instruction Set Computers.     

基于网格的神经网络计算平台的架构

介绍了神经网络计算平台在网格上的架构。由于神经网络应用环境的复杂性及要求处理大量数据的问题,从而神经网络计算需要超强的计算能力,因此在神经网络计算平台中引入了网格的思想,力图建立一个基于网格的神经网络计算平台。平台使用Globus工具,采用了统一控制和完全托管的思想。鉴于神经网络算法的复杂性和大数据量,神经计算算法在网格上进行适合各自特性的分解。     

Predicting stock index increments by neural networks: The role of trading volume under different horizons

Recent studies show that there is a significant bidirectional nonlinear causality between stock return and trading volume. In this research, we reinforce this statement and the results presented in some earlier literatures and further investigate whether trading volume can significantly improve the prediction performance of neural networks under short-, medium-and long-term forecasting horizons. An application of component-based neural networks is used in forecasting one-step ahead stock index increments. The models are also augmented by the addition of different combinations of indices’ and component stocks’ trading volumes as inputs to form more general ex-ante forecasting models. Neural networks are trained with the data of stock returns and volumes from NASDAQ, DJIA and STI indices. Results indicate that augmented neural network models with trading volumes lead to improvements, at different extents, in forecasting performance under different terms of forecasting horizon. Empirical results indicate that trading volumes lead to modest improvements on the performance of stock index increments prediction under medium-and long-term horizons.     

一类高阶Cohen-Grossberg神经网络的多周期性分析

利用状态空间分解方法,探讨一类具有特殊激励函数的高阶Ｃｏｈｅｎ-Ｇｒｏｓｓｂｅｒｇ神经网络的多周期性问题．该类神经网络的激励函数包括带有饱和区的非递减函数以及一般的细胞神经网络激励函数等．给出了保证此类网络的周期环在饱和区内局部指数收敛的充分条件．所得结果表明,一个狀维网络可以有２ⁿ个局部指数收敛的周期环存在于饱和区．最后以一个数值例子说明了所得结果的有效性．     

Drought estimation with neural networks

An artificial neural network model is presented to derive streamflow precipitation data. It is tested with actual data coming from a nearby river, referred to a basin area of 356 km² and a time period of 11 years. A feedforward multilayer perception with linear output has been built to deal with this problem. The dynamics are caught by the filter structure of the input layer.

A special study on crossing properties, based on training sample selection,is made to measure the performance of the network for drought analysis. Sample selection leads to increased accuracy within the sample range and degraded performance for points that are clearly out. Predicted number of droughts, average drought length and deficit are compared with the actual data. The results show that very simple neural network models can give fine results.     

Fuzzy rough granular neural networks, fuzzy granules, and classification

We introduce a fuzzy rough granular neural network (FRGNN) model based on the multilayer perceptron using a back-propagation algorithm for the fuzzy classification of patterns. We provide the development strategy of the network mainly based upon the input vector, initial connection weights determined by fuzzy rough set theoretic concepts, and the target vector. While the input vector is described in terms of fuzzy granules, the target vector is defined in terms of fuzzy class membership values and zeros. Crude domain knowledge about the initial data is represented in the form of a decision table, which is divided into subtables corresponding to different classes. The data in each decision table is converted into granular form. The syntax of these decision tables automatically determines the appropriate number of hidden nodes, while the dependency factors from all the decision tables are used as initial weights. The dependency factor of each attribute and the average degree of the dependency factor of all the attributes with respect to decision classes are considered as initial connection weights between the nodes of the input layer and the hidden layer, and the hidden layer and the output layer, respectively. The effectiveness of the proposed FRGNN is demonstrated on several real-life data sets.     

Predicting peak pinch strength: Artificial neural networks vs. regression

Cumulative trauma disorders (CTDs) of the upper extremities are one of the major ergonomics areas of research. Pinching is a common risk factor associated with the development of hand/wrist CTDs. The capacity standards of peak pinch strength for various postures are needed to design the tasks in harmony with the workers. This paper describes the formulation, building and comparison of pinch strength prediction models that were obtained using two approaches: Statistical and artificial neural networks (ANN). Statistical and ANN models were developed to predict the peak chuck pinch strength as a function of different combinations of five elbow and seven shoulder flexion angles, and several anthropometric and physiological variables. The two modeling approaches were compared. The results indicate ANN models to provide more accurate predictions over the standard statistical models.