Self-organizing neural networks for the analysis and representation of data: Some financial cases

Many recent papers have dealt with the application of feedforward neural networks in financial data processing. This powerful neural model can implement very complex nonlinear mappings, but when outputs are not available or clustering of patterns is required, the use of unsupervised models such as self-organizing maps is more suitable. The present work shows the capabilities of self-organizing feature maps for the analysis and representation of financial data and for aid in financial decision-making. For this purpose, we analyse the Spanish banking crisis of 1977–1985 and the Spanish economic situation in 1990 and 1991, making use of this unsupervised model. Emphasis is placed on the analysis of the synaptic weights, fundamental for delimiting regions on the map, such as bankrupt or solvent regions, where similar companies are clustered. The time evolution of the companies and other important conclusions can be drawn from the resulting maps.Characters and symbols used and their meaning nx x dimension of the neuron grid, in number of neurons - ny y dimension of the neuron grid, in number of neurons - n dimension of the input vector, number of input variables - (i, j) indices of a neuron on the map - k index of the input variables - w _ijk synaptic weight that connects thek input with the (i, j) neuron on the map - W _ij weight vector of the (i, j) neuron - x _k input vector - X input vector - (t) learning rate - _o starting learning rate - _f final learning rate - R(t) neighbourhood radius - R₀ starting neighbourhood radius - R _f final neighbourhood radius - t iteration counter - t _rf number of iterations until reachingR _f - t _f number of iterations until reaching _f - h(·) lateral interaction function - standard deviation - for every - d (x, y) distance between the vectors x and y     

Symbolic representation of neural networks

Neural networks often surpass decision trees in predicting pattern classifications, but their predictions cannot be explained. This algorithm's symbolic representations make each prediction explicit and understandable. Our approach to understanding a neural network uses symbolic rules to represent the network decision process. The algorithm, NeuroRule, extracts these rules from a neural network. The network can be interpreted by the rules which, in general, preserve network accuracy and explain the prediction process. We based NeuroRule on a standard three layer feed forward network. NeuroRule consists of four phases. First, it builds a weight decay backpropagation network so that weights reflect the importance of the network's connections. Second, it prunes the network to remove irrelevant connections and units while maintaining the network's predictive accuracy. Third, it discretizes the hidden unit activation values by clustering. Finally, it extracts rules from the network with discretized hidden unit activation values     

A methodology for building neural networks models from empirical engineering data

Neural networks (NN) are general tools for modeling functional relationships in engineering. They are used to model the behavior of products and the properties of processes. Nevertheless, their use is often ad hoc. This paper provides a sound basis for using NN as tools for modeling functional relationships implicit in empirical engineering data. First, a clear definition of a modeling task is given, followed by reviewing the theoretical modeling capabilities of NN and NN model estimation. Subsequently, a procedure for using NN in engineering practice is described and illustrated with an example of modeling marine propeller behavior. Particular attention is devoted to better estimation of model quality, insight into the influence of measurement errors on model quality, and the use of advanced methods such as stacked generalization and ensemble modeling to further improve model quality. Using a new method of ensemble of SG(k-NN), one could improve the quality of models even if they are close to being optimal.     

Developing empirical models from observational data using artificial neural networks

Computer-integrated manufacturing requires models of manufacturing processes to be implemented on the computer. Process models are required for designing adaptive control systems and selecting optimal parameters during process planning. Mechanistic models developed from the principles of machining science are useful for implementing on a computer. However, in spite of the progress being made in mechanistic process modeling, accurate models are not yet available for many manufacturing processes. Empirical models derived from experimental data still play a major role in manufacturing process modeling. Generally, statistical regression techniques are used for developing such models. However, these techniques suffer from several disadvantages. The structure (the significant terms) of the regression model needs to be decided a priori. These techniques cannot be used for incrementally improving models as new data becomes available. This limitation is particularly crucial in light of the advances in sensor technology that allow economical on-line collection of manufacturing data. In this paper, we explore the use of artificial neural networks (ANN) for developing empirical models from experimental data for a machining process. These models are compared with polynomial regression models to assess the applicability of ANN as a model-building tool for computer-integrated manufacturing.Operated for the United States Department of Energy under contract No. DE-AC04-76-DP00613.     

Cross-model convolutional neural network for multiple modality data representation

Neural Computing and Applications - A novel data representation method of convolutional neural network (CNN) is proposed in this paper to represent data of different modalities. We learn a CNN...     

基于卷积神经网络的图像数据增强算法

提升卷积神经网络的泛化能力和降低过拟合的风险是深度卷积神经网络的研究重点。遮挡是影响卷积神经网络泛化能力的关键因素之一,通常希望经过复杂训练得到的模型能够对遮挡图像有良好的泛化性。为了降低模型过拟合的风险和提升模型对随机遮挡图像识别的鲁棒性,提出了激活区域处理算法,在训练过程中对某一卷积层的最大激活特征图进行处理后对输入图像进行遮挡,然后将被遮挡的新图像作为网络的新输入并继续训练模型。实验结果表明,提出的算法能够提高多种卷积神经网络模型在不同数据集上的分类性能,并且训练好的模型对随机遮挡图像的识别具有非常好的鲁棒性。     

Cognitive engineering based knowledge representation in neural networks

A model of a human neural knowledge processing system is presented that suggests the following. First, an entity in the outside world lends to be locally encoded in neural networks so that the conceptual information structure is mirrored in its physical implementation. Second, the knowledge of problem solving is implemented in a quite implicit way in the internal structure of the neural network (a functional group of associated hidden neurons and their connections to entity neurons) not in individual neurons or connections. Third, the knowledge system is organized and implemented in a modular fashion in neural networks according to the local specialization of problem solving where a module of neural network implements an inter-related group of knowledge such as a schema, and different modules have similar processing mechanisms, but differ in their input and output patterns. A neural network module can be tuned just as a schema structure can be adapted for changing environments. Three experiments were conducted to try to validate the suggested cognitive engineering based knowledge structure in neural networks through computer simulation. The experiments, which were based on a task of modulo arithmetic, provided some insights into the plausibility of the suggested model of a neural knowledge processing system.     

Cognitive engineering based knowledge representation in neural networks

Abstract

A model of a human neural knowledge processing system is presented that suggests the following. First, an entity in the outside world lends to be locally encoded in neural networks so that the conceptual information structure is mirrored in its physical implementation. Second, the knowledge of problem solving is implemented in a quite implicit way in the internal structure of the neural network (a functional group of associated hidden neurons and their connections to entity neurons) not in individual neurons or connections. Third, the knowledge system is organized and implemented in a modular fashion in neural networks according to the local specialization of problem solving where a module of neural network implements an inter-related group of knowledge such as a schema, and different modules have similar processing mechanisms, but differ in their input and output patterns. A neural network module can be tuned just as a schema structure can be adapted for changing environments. Three experiments were conducted to try to validate the suggested cognitive engineering based knowledge structure in neural networks through computer simulation. The experiments, which were based on a task of modulo arithmetic, provided some insights into the plausibility of the suggested model of a neural knowledge processing system.     

Common-input models for multiple neural spike-train data

Recent developments in multi-electrode recordings enable the simultaneous measurement of the spiking activity of many neurons. Analysis of such multineuronal data is one of the key challenge in computational neuroscience today. In this work, we develop a multivariate point-process model in which the observed activity of a network of neurons depends on three terms: (1) the experimentally-controlled stimulus; (2) the spiking history of the observed neurons; and (3) a hidden term that corresponds, for example, to common input from an unobserved population of neurons that is presynaptic to two or more cells in the observed population. We consider two models for the network firing-rates, one of which is computationally and analytically tractable but can lead to unrealistically high firing-rates, while the other with reasonable firing-rates imposes a greater computational burden. We develop an expectation-maximization algorithm for fitting the parameters of both the models. For the analytically tractable model the expectation step is based on a continuous-time implementation of the extended Kalman smoother, and the maximization step involves two concave maximization problems which may be solved in parallel. The other model that we consider necessitates the use of Monte Carlo methods for the expectation as well as maximization step. We discuss the trade-off involved in choosing between the two models and the associated methods. The techniques developed allow us to solve a variety of inference problems in a straightforward, computationally efficient fashion; for example, we may use the model to predict network activity given an arbitrary stimulus, infer a neuron's ring rate given the stimulus and the activity of the other observed neurons, and perform optimal stimulus decoding and prediction. We present several detailed simulation studies which explore the strengths and limitations of our approach.     

Iconic representation of visual data and models

Knowledge based image analysis is a combination of digital signal processing and symbolic reasoning. In this paper, we will look at some problems connected to the symbolic reasoning approach to image interpretation and see how an iconic representation can help to solve some of them. We will show that many of the features and problems connected with both symbolic and iconic representation are complementary.     

Application of fuzzy neural networks for the data clusterization problem

The fuzzy neural network for pattern clusterization is described. The network training is based on c-means algorithm. An original methodology for the statistical interpretation of result of c-means clusterization is proposed. As an example, the clusterization of vessel hull tenzo measurements on the arctic route. An approach to preprocessing of the raw measurements for the further usage by means of the described methods is discussed. The text was submitted by the author in English.     

A neural networks algorithm for data path synthesis

This paper presents a deterministic parallel algorithm to solve the data path allocation problem in high-level synthesis. The algorithm is driven by a motion equation that determines the neurons firing conditions based on the modified Hopfield neural network model of computation. The method formulates the allocation problem using the clique partitioning problem, an NP-complete problem, and handles multicycle functional units as well as structural pipelining. The algorithm has a running time complexity of O(1) for a circuit with n operations and c shared resources. A sequential simulator was implemented on a Linux Pentium PC under X-Windows. Several benchmark examples have been implemented and favorable design comparisons to other synthesis systems are reported.     

An alternative approach for modelling and simulation of traffic data: artificial neural networks

It is assumed that there is a complicated relationship between the driver characteristics and involvement in traffic accidents. It is quite difficult to simulate the effects of these driver characteristics into the traffic accidents. The artificial neural networks (ANN) approach is proposed for training-predicting the database in this paper since it is a more flexible and assumption-free methodology. The networks are organised in different architectures and the results have been compared in order to determine the best fitting one. Finally, the best possible architecture is selected for a better representation of the survey data and the prediction of accident percentage. The predictions about the outputs for the inputs which are not used in the training of the ANN provide information about the drivers which cannot be reached in the database. The predictions are highly satisfactory and the ANNs have been found to be reliable processing systems for modelling and simulation in the traffic data assessments.     

Time-delay neural networks: representation and induction offinite-state machines

In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNNs) with input delay neural networks (IDNNs), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMMs), a subclass of finite-state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis.     

An educational tool for artificial neural networks

Artificial neural networks are some kind of data processing systems, which try to simulate features of the human brain and its learning process. So, they are widely used by researchers to solve different problems in optimization, classification, pattern recognition, associative memory and control. In this paper, an educational tool, which can be used to work on different kinds of neural network models and learn fundamentals of the artificial neural network, is described. At this point, the whole tool environment provides an advanced system to ensure mentioned functions. The developed system supports using MLP, LVQ and SOM models and related learning algorithms. It employs some visual, interactive tools, which enable users to compose their own neural networks and work on the developed networks easily. By using these tools, users can also understand and learn working mechanism of a typical artificial neural network, using features of different models and related learning algorithms.     

Exploiting weak mask representation with convolutional neural networks for accurate object tracking

Multimedia Tools and Applications - Recent years have witnessed the popularity of Convolutional Neural Networks (CNN) in a variety of computer vision tasks, including video object tracking....     

MLP-based adaptive neural control of nonlinear time-delay systems with the unknown hysteresis

In this note, the authors study the tracking problem for uncertain nonlinear time-delay systems with unknown non-smooth hysteresis described by the generalised Prandtl–Ishlinskii (P-I) model. A minimal learning parameters (MLP)-based adaptive neural algorithm is developed by fusion of the Lyapunov–Krasovskii functional, dynamic surface control technique and MLP approach without constructing a hysteresis inverse. Unlike the existing results, the main innovation can be summarised as that the proposed algorithm requires less knowledge of the plant and independent of the P-I hysteresis operator, i.e. the hysteresis effect is unknown for the control design. Thus, the outstanding advantage of the corresponding scheme is that the control law is with a concise form and easy to implement in practice due to less computational burden. The proposed controller guarantees that the tracking error converges to a small neighbourhood of zero and all states of the closed-loop system are stabilised. A simulation example demonstrates the effectiveness of the proposed scheme.     

Effective data mining using neural networks

Classification is one of the data mining problems receiving great attention recently in the database community. The paper presents an approach to discover symbolic classification rules using neural networks. Neural networks have not been thought suited for data mining because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. With the proposed approach, concise symbolic rules with high accuracy can be extracted from a neural network. The network is first trained to achieve the required accuracy rate. Redundant connections of the network are then removed by a network pruning algorithm. The activation values of the hidden units in the network are analyzed, and classification rules are generated using the result of this analysis. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of standard data mining test problems