Financial decision support using neural networks and support vector machines

Abstract: Bankruptcy prediction and credit scoring are the two important problems facing financial decision support. The multilayer perceptron (MLP) network has shown its applicability to these problems and its performance is usually superior to those of other traditional statistical models. Support vector machines (SVMs) are the core machine learning techniques and have been used to compare with MLP as the benchmark. However, the performance of SVMs is not fully understood in the literature because an insufficient number of data sets is considered and different kernel functions are used to train the SVMs. In this paper, four public data sets are used. In particular, three different sizes of training and testing data in each of the four data sets are considered (i.e. 3:7, 1:1 and 7:3) in order to examine and fully understand the performance of SVMs. For SVM model construction, the linear, radial basis function and polynomial kernel functions are used to construct the SVMs. Using MLP as the benchmark, the SVM classifier only performs better in one of the four data sets. On the other hand, the prediction results of the MLP and SVM classifiers are not significantly different for the three different sizes of training and testing data.     

Synaptic weight noise during multilayer perceptron training: faulttolerance and training improvements

The authors develop a mathematical model of the effects of synaptic arithmetic noise in multilayer perceptron training. Predictions are made regarding enhanced fault-tolerance and generalization ability and improved learning trajectory. These predictions are subsequently verified by simulation. The results are perfectly general and have profound implications for the accuracy requirements in multilayer perceptron (MLP) training, particularly in the analog domain.     

Single imputation with multilayer perceptron and multiple imputation combining multilayer perceptron and k-nearest neighbours for monotone patterns

The knowledge discovery process is supported by data files information gathered from collected data sets, which often contain errors in the form of missing values. Data imputation is the activity aimed at estimating values for missing data items. This study focuses on the development of automated data imputation models, based on artificial neural networks for monotone patterns of missing values. The present work proposes a single imputation approach relying on a multilayer perceptron whose training is conducted with different learning rules, and a multiple imputation approach based on the combination of multilayer perceptron and k-nearest neighbours. Eighteen real and simulated databases were exposed to a perturbation experiment with random generation of monotone missing data pattern. An empirical test was accomplished on these data sets, including both approaches (single and multiple imputations), and three classical single imputation procedures – mean/mode imputation, regression and hot-deck – were also considered. Therefore, the experiments involved five imputation methods. The results, considering different performance measures, demonstrated that, in comparison with traditional tools, both proposals improve the automation level and data quality offering a satisfactory performance.     

A class of hybrid multilayer perceptrons for software development effort estimation problems

The dilation-erosion-linear perceptron is a hybrid morphological neuron which has been recently proposed in the literature to solve some prediction problems. However, a drawback arises from such model for building mappings to solve tasks with complex input-output nonlinear relationships within effort estimation problems. In this sense, to overcome this limitation, we present a particular class of hybrid multilayer perceptrons, called the multilayer dilation-erosion-linear perceptron (MDELP), to deal with software development effort estimation problems. Each processing unit of the proposed model is composed of a mix between a hybrid morphological operator (given by a balanced combination between dilation and erosion operators) and a linear operator. According to Pessoa and Maragos’s ideas, we propose a descending gradient-based learning process to train the proposed model. Besides, we conduct an experimental analysis using relevant datasets of software development effort estimation and the achieved results are discussed and compared, according to MMRE and PRED25 measures, to those obtained by classical and state of the art models presented in the literature.     

Automatic feature selection for supervised learning in link prediction applications: a comparative study

For the last years, a considerable amount of attention has been devoted to the research about the link prediction (LP) problem in complex networks. This problem tries to predict the likelihood of an association between two not interconnected nodes in a network to appear in the future. One of the most important approaches to the LP problem is based on supervised machine learning (ML) techniques for classification. Although many works have presented promising results with this approach, choosing the set of features (variables) to train the classifiers is still a major challenge. In this article, we report on the effects of three different automatic variable selection strategies (Forward, Backward and Evolutionary) applied to the feature-based supervised learning approach in LP applications. The results of the experiments show that the use of these strategies does lead to better classification models than classifiers built with the complete set of variables. Such experiments were performed over three datasets (Microsoft Academic Network, Amazon and Flickr) that contained more than twenty different features each, including topological and domain-specific ones. We also describe the specification and implementation of the process used to support the experiments. It combines the use of the feature selection strategies, six different classification algorithms (SVM, K-NN, naïve Bayes, CART, random forest and multilayer perceptron) and three evaluation metrics (Precision, F-Measure and Area Under the Curve). Moreover, this process includes a novel ML voting committee inspired approach that suggests sets of features to represent data in LP applications. It mines the log of the experiments in order to identify sets of features frequently selected to produce classification models with high performance. The experiments showed interesting correlations between frequently selected features and datasets.     

Online State–Space Modeling Using Recurrent Multilayer Perceptrons with Unscented Kalman Filter

A nonlinear black-box modeling approach using a state–space recurrent multilayer perceptron (RMLP) is considered in this paper. The unscented Kalman filter (UKF), which was proposed recently and is appropriate for state–space representation, is employed to train the RMLP. The UKF offers a derivative-free computation and an easy implementation, compared to the extended Kalman filter (EKF) widely used for training neural networks. In addition, the UKF has a fast convergence rate and an excellent capability of parameter estimation which are appropriate for online learning. Through modeling experiments of nonlinear systems, the effectiveness of the RMLP trained with the UKF is demonstrated.     

基于深度信念网络的异常点集间的匹配算法

在存在异常值、噪声或缺失点的情况下,损坏的点集中很难区分异常点与正常点,并且点集之间的匹配关系也会受到这些异常点的影响。基于正常点之间存在某种联系以及正常点与异常点之间存在差异的先验知识,提出将点集间匹配关系的估计问题模型化为机器学习的过程。首先,考虑到两个正常点集之间的误差特征,提出了一种基于深度信念网络（DBN）的学习方法来训练具有正常点集的网络;然后,使用训练好的DBN测试损坏的点集,根据设置的误差阈值在网络输出端就可以识别异常值和不匹配的点。对存在噪声和缺失点的2D、3D点集所做的匹配实验中,利用模型预测样本的结果定量评估了点集间的匹配性能,其中匹配的精确率可以达到94%以上。实验结果表明,所提算法可以很好地检测点集中的噪声,即使在数据缺失的情况下,该算法也可以识别几乎所有的匹配点。     

BRAIN EMOTIONAL LEARNING-BASED PATTERN RECOGNIZER

In this article, the brain emotional learning-based pattern recognizer (BELPR) is proposed to solve multiple input–multiple output classification and chaotic time series prediction problems. BELPR is based on an extended computational model of the human brain limbic system that consists of an emotional stimuli processor. The BELPR is model free and learns the patterns in a supervised manner and evaluates the output(s) using the activation function tansig. In the numerical studies, various comparisons are made between BELPR and a multilayer perceptron (MLP) with a back-propagation learning algorithm. The methods are tested to classify 12 UCI (University of California, Irvine) machine learning data sets and to predict activity indices of the Earth's magnetosphere. The main features of BELPR are higher accuracy, decreased time and spatial complexity, and faster training.     

人工神经网络BP算法的改进及其在无损检测中的应用

采用多层感知器（MLP）与误差反向传播算法（er-ror back-propagation algorithm)构造与监督训练人工祖辈 经网络,采用了改进的非线性激励函数与学习率的误差反向传播算法,超声无损检测的计算机模拟与实验结果表明,改进的BP算法收敛速度较之BP算法明显加快。     

Two algorithms for neural-network design and training withapplication to channel equalization

We describe two algorithms for designing and training neural-network classifiers. The first, the linear programming slab algorithm (LPSA), is motivated by the problem of reconstructing digital signals corrupted by passage through a dispersive channel and by additive noise. It constructs a multilayer perceptron (MLP) to separate two disjoint sets by using linear programming methods to identify network parameters. The second, the perceptron learning slab algorithm (PLSA), avoids the computational costs of linear programming by using an error-correction approach to identify parameters. Both algorithms operate in highly constrained parameter spaces and are able to exploit symmetry in the classification problem. Using these algorithms, we develop a number of procedures for the adaptive equalization of a complex linear 4-quadrature amplitude modulation (QAM) channel, and compare their performance in a simulation study. Results are given for both stationary and time-varying channels, the latter based on the COST 207 GSM propagation model     

Classification of linearly nonseparable patterns by linearthreshold elements

Learning and convergence properties of linear threshold elements or perceptrons are well understood for the case where the input vectors (or the training sets) to the perceptron are linearly separable. Little is known, however, about the behavior of the perceptron learning algorithm when the training sets are linearly nonseparable. We present the first known results on the structure of linearly nonseparable training sets and on the behavior of perceptrons when the set of input vectors is linearly nonseparable. More precisely, we show that using the well known perceptron learning algorithm, a linear threshold element can learn the input vectors that are provably learnable, and identify those vectors that cannot be learned without committing errors. We also show how a linear threshold element can be used to learn large linearly separable subsets of any given nonseparable training set. In order to develop our results, we first establish formal characterizations of linearly nonseparable training sets and define learnable structures for such patterns. We also prove computational complexity results for the related learning problems. Next, based on such characterizations, we show that a perceptron does the best one can expect for linearly nonseparable sets of input vectors and learns as much as is theoretically possible.     

基于改进的深度置信网络的电离层F2层临界频率预测

随着深度学习理论的日趋成熟,针对各种场合的实际应用逐渐成为研究的焦点。本文提出一种基于深度置信网络（Deep Belief Network,DBN）对本区域未来24h的电离层临界频率foF2进行预测的方法。首先,对选取的数据集进行处理,生成用于训练和测试的数据集,其次改进DBN网络基本单元的结构,以适应对连续型数据特征的提取与学习,再通过试验的方式确定DBN网络基本结构,最后利用训练数据集对改进后的网络进行训练,实现对 foF2值的预测。与实测值相比较,改进的DBN网络具有极佳的预测准确性,与浅层结构BP网络和SVM网络相比,改进的DBN网络不但克服了浅层结构所固有的问题,而且也表现出更加优异的对于连续型数据的预测性能,尤其是当预测值受到高维复杂因素影响时改进的DBN模型依旧能表现出很好的预测性能。     

Divide-and-conquer learning and modular perceptron networks

A novel modular perceptron network (MPN) and divide-and-conquer learning (DCL) schemes for the design of modular neural networks are proposed. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region into two easier to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We evaluated and compared the proposed MPN with several representative neural networks on the two-spirals problem and real-world dataset. The MPN achieved better weight learning performance by requiring much less data presentations during the network training phases, and better generalization performance, and less processing time during the retrieving phase.     

Developing an expert system for predicting alluvial channel geometry using ANN

Cross section geometry of stable alluvial channels usually is estimated by simple inaccurate empirical equations, because of the complexity of the phenomena and unknown physical processes of regime channels. So, the main purpose of this study is to evaluate the potential of simulating regime channel treatments using artificial neural networks (ANNs). The process of training and testing of this new model is done using a set of available published filed data (371 data numbers). Several statistical and graphical criterions are used to check the accuracy of the model in comparison with previous empirical equations. The multilayer perceptron (MLP) artificial neural network was used to construct the simulation model based on the training data using back-propagation algorithm. The results show a considerably better performance of the ANN model over the available empirical or rational equations. The constructed ANN models can almost perfectly simulate the width, depth and slope of alluvial regime channels, which clearly describes the dominant geometrical parameters of alluvial rivers. The results demonstrate that the ANN can precisely simulate the regime channel geometry, while the empirical, regression or rational equations can’t do this. The presented methodology in this paper is a new approach in establishing alluvial regime channel relations and predicting cross section geometry of alluvial rivers also it can be used to design stable irrigation and water conveyance channels.     

Two efficient connectionist schemes for structure preservingdimensionality reduction

We propose two neural net based methods for structure preserving dimensionality reduction. Method 1 selects a small representative sample and applies Sammon's method to project it. This projected data set is then used to train a multilayer perceptron (MLP). Method 2 uses Kohonen's self-organizing feature map to generate a small set of prototypes which is then projected by Sammon's method. This projected data set is then used to train an MLP. Both schemes are quite effective in terms of computation time and quality of output, and both outperform methods of Jain and Mao (1992, 1995) on the data sets tried.     

基于半监督学习和支持向量机的煤与瓦斯突出预测研究

针对支持向量机要求输入向量为已标记样本,而实际应用中已标记样本很难获取的问题,提出将半监督学习和支持向量机结合的煤与瓦斯突出预测方法;介绍了采用SVM预测煤与瓦斯突出的流程及其输入向量的选择;对半监督学习中的协同训练算法进行了改进:在同一属性集上训练2个不同分类器SVM和KNN,将2个分类器标记一致的样本加入训练集,从而充分利用未标记样本不断补充信息,更新训练集标记样本,达到强化训练集的目的。测试结果表明,改进后的算法比单独的支持向量机预测方法准确率更高。     

基于人工神经网络的软件失效预测模型研究

在软件开发的早期预测有失效倾向的软件模块,能够极大的提高软件的质量。软件失效预测中的一个普遍的问题是数据中存在噪声,而神经网络具有鲁棒性并对噪声有很强的抑制能力。该文介绍了一种基于人工神经网络的软件失效预测模型,给出了基于反向传播算法的多层前向网络的网络结构。用这种方法对朗讯光网络有限公司开发的SDH通信软件进行了分析,并得到了较高的预测准确率。通过采集通信软件的不同发布版本的测试历史数据,讨论了训练集数据的选择与预测精度之间的关系。     

基于改进多层感知机模型的港口吞吐量预测研究

精确的港口货物吞吐量预测对于港口的发展至关重要。本文提出了改进粒子群优化去尾均值多层感知机模型对上海港货物吞吐量进行预测。选取了影响上海港货物吞吐量的十个因素进行训练,实验结果表明该预测模型的预测性能明显优于传统MLP预测模型和基本的粒子群优化多层感知机模型。对该预测模型的误差分析和收敛性分析表明该预测模型可靠。     

Localized generalization error model and its application to architecture selection for radial basis function neural network

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machine (SVM), radial basis function neural network (RBFNN), and multilayer perceptron neural network (MLPNN) are local learning machines for solving problems and treat unseen samples near the training samples to be more important. In this paper, we propose a localized generalization error model which bounds from above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure. It is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments using 17 University of California at Irvine (UCI) data sets show that, in comparison with cross validation (CV), sequential learning, and two other ad hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.     

k-nearest neighbors directed noise injection in multilayerperceptron training

The relation between classifier complexity and learning set size is very important in discriminant analysis. One of the ways to overcome the complexity control problem is to add noise to the training objects, increasing in this way the size of the training set. Both the amount and the directions of noise injection are important factors which determine the effectiveness for classifier training. In this paper the effect is studied of the injection of Gaussian spherical noise and k-nearest neighbors directed noise on the performance of multilayer perceptrons. As it is impossible to provide an analytical investigation for multilayer perceptrons, a theoretical analysis is made for statistical classifiers. The goal is to get a better understanding of the effect of noise injection on the accuracy of sample-based classifiers. By both empirical as well as theoretical studies, it is shown that the k-nearest neighbors directed noise injection is preferable over the Gaussian spherical noise injection for data with low intrinsic dimensionality.