A practical overview of neural networks

This paper overviews the myths and misconceptions that have surrounded neural networks in recent years. Focusing on backpropagation and the Hopfield network, we discuss the problems that have plagued practical application of these techniques, and review some of the recent progress made. Both real and perceived inadequacies of backpropagation are discussed, as well as the need for an understanding of statistics and of the problem domain in order to apply and assess the neural network properly. We consider alternatives or variants to backpropagation, which overcome some of its real limitations. The Hopfield network's poor performance on the traveling salesman problem in combinatorial optimization has colored its reception by engineers; we describe both new research in this area and promising results in other practical optimization applications. Overall, it is hoped, this paper will aid in a more balanced understanding of neural networks. They seem worthy of consideration in many applications, but they do not deserve the status of a panacea – nor are they as fraught with problems as would now seem to be implied.     

Predicting Aflatoxin Contamination in Peanuts: A Genetic Algorithm/Neural Network Approach

Aflatoxin contamination in peanut crops is a problem of significant health and financial importance. Predicting aflatoxin levels prior to crop harvest is useful for minimizing the impact of a contaminated crop and is the goal of our research. Backpropagation neural networks have been used to model problems of this type, however development of networks poses the complex problem of setting values for architectural features and backpropagation parameters. Genetic algorithms have been used in other studies to determine parameters for backpropagation neural networks. This paper describes the development of a genetic algorithm/backpropagation neural network hybrid (GA/BPN) in which a genetic algorithm is used to find architectures and backpropagation parameter values simultaneously for a backpropagation neural network that predicts aflatoxin contamination levels in peanuts based on environmental data. Learning rate, momentum, and number of hidden nodes are the parameters that are set by the genetic algorithm. A three-layer feed-forward network with logistic activation functions is used. Inputs to the network are soil temperature, drought duration, crop age, and accumulated heat units. The project showed that the GA/BPN approach automatically finds highly fit parameter sets for backpropagation neural networks for the aflatoxin problem.     

Using neural networks to predict financial reporting fraud: Part 2

Artificial Intelligence (AI) is the science of building computer devices and software applications that mimic many of the characteristics that we associate with human behaviour, such as the ability to reason, see, learn, solve problems, understand language and so on. AI systems include natural languages, robotics, expert systems and neural networks. This article focuses on neural networks, which can be designed for problems that arise at any managerial level and may be used to predict the occurrence of fraud.     

Integrated feature architecture selection

In this paper, we present an integrated approach to feature and architecture selection for single hidden layer-feedforward neural networks trained via backpropagation. In our approach, we adopt a statistical model building perspective in which we analyze neural networks within a nonlinear regression framework. The algorithm presented in this paper employs a likelihood-ratio test statistic as a model selection criterion. This criterion is used in a sequential procedure aimed at selecting the best neural network given an initial architecture as determined by heuristic rules. Application results for an object recognition problem demonstrate the selection algorithm's effectiveness in identifying reduced neural networks with equivalent prediction accuracy.     

Using neural networks to predict financial reporting fraud: Part 1

Artificial Intelligence (AI) is the science of building computer devices and software applications that mimic many of the characteristics that we associate with human behaviour, such as the ability to reason, see, learn, solve problems, understand languages, and so on. AI systems include natural languages, robotics, expert systems and neural networks. This article focuses on neural networks, which can be designed for problems that arise at any managerial level and may be used to predict the occurrence of fraud.     

Analysis of a deformed three-dimensional culvert structure using neural networks

Applying dynamic backpropagation neural networks with energy function as minimization index, the deformed behaviors for culvert structure under a static loading are analyzed in this paper. The training process is avoided by using stiffness matrix and force vector of the structure instead of using weighting matrix and bias vector in the neural networks calculations. The ability of neural networks is verified by comparing the results with analytical solutions and finite element solutions. In order to improve the numerical accuracy, three grid systems are used to model the problem and to check the grid independence. From the concept of energy, the existence of an attractor for the three grid systems is proved and the solution is obtained accordingly. In addition, from the numerical experiments, the convergence rate can be accelerated significantly by introducing a relaxation factor in the calculation. Based on the displacement profile and the three-dimensional displacement plot, the results reasonably show that more downward deformations occur at the centerline of the whole culvert structure, particularly at the top surface of the centerline. The obtained information may provide a better understanding of typical structural problems frequently found in the field of civil engineering.     

一种改进的前馈神经网络结构及其在复杂系统建模中的应用

针对包含多道加工工序、输入变量很多的复杂工业系统建模精度难以提高的问题,提出一种改进的前馈神经网络结构,输入变量不是由同一层输入,而是根据变量起作用的前后次序分别在网络的不同层输入,真实反映了大工业过程的各生产工序中的参数发生作用的时间顺序。同时由于输入变量在适当的时候输入网络,从而使网络的规模减小。该神经网络是处理高维问题,尤其是建立包含多道加工工序的大工业过程模型问题的强有力工具。将该神经网络用于热连轧产品质量建模,经过实测数据拟合与检验,仿真结果表明:提出的小波神经网络结构是可行的而且有很好的应用前景。     

An expert system based on wavelet decomposition and neural network for modeling Chua’s circuit

This paper presents an expert system based on wavelet decomposition and neural network for modeling and simulation of Chua’s circuit which is used for chaos studies. The problems which arise in modeling Chua’s circuit by neural networks are high structural complexity and slow and difficult training. With this proposed method a new solutions is produced to solve these problems. Wavelet decomposition is used for new useful feature extracting from input signal and neural network is used for modeling. Test results of proposed wavelet decomposition and neural network model are compared with test results of neural network model. Desired performance is provided by this new model. Test results showed that the suggested method can be used efficiently for modeling nonlinear dynamical systems.     

Inference, inquiry, evidence censorship, and explanation inconnectionist expert systems

The combination of the techniques of expert systems and neural networks has the potential of producing more powerful systems, for example, expert systems able to learn from experience. In this paper, we address the combinatorial neural model (CNM), a kind of fuzzy neural network able to accommodate in a simple framework the highly desirable property of incremental learning, as well as the usual capabilities of expert systems. We show how an interval-based representation for membership grades makes CNM capable of reasoning with several types of uncertainty: vagueness, ignorance, and relevance commonly found in practical applications. In addition, we show how basic functions of expert systems such as inference, inquiry, censorship of input information, and explanation may be implemented. We also report experimental results of the application of CNM to the problem of deforestation monitoring of the Amazon region using satellite images     

Unified problem modeling language for knowledge engineering of complex systems

The 90s has seen the emergence of hybrid configurations of four most commonly used intelligent methodologies, namely, symbolic knowledge based systems (e.g. expert systems), artificial neural networks, fuzzy systems and genetic algorithms. These hybrid configurations are used for different problem solving tasks/situations. In this paper we describe unified problem modeling language at two different levels, the task structure level for knowledge engineering of complex data intensive domains, and the computational level of the task level hybrid architecture. Among other aspects, the unified problem modeling language considers various intelligent methodologies and their hybrid configurations as technological primitives used to accomplish various tasks defined at the task structure level. The unified problem modeling language is defined in the form of five problem solving adapters. The problem solving adapters outline the goals, tasks, percepts/inputs, and hard and soft computing methods for modeling complex problems. The task structure level has been applied in modeling several applications in e-commerce, image processing, diagnosis, and other complex, time critical, and data intensive domains. We also define a layered intelligent multi-agent, operating system processes, intelligent technologies with the task structure level associative hybrid architecture. The layered architecture also facilitates component based software modeling process.Work Supported by VPAC grant no EPPNLA002.2001     

Neural networks and adaptive expert systems in the CSA approach

According to many authors, neural networks and adaptive expert systems may provide the foundations of sixth-generation computers. Neural networks use lower hardware-like concepts and they are based on continuous and numeric type computation. On the other hand, adaptive expert systems use inference rules and perform high-level symbolic computations. the approaches may seem to be totally different, but they do exhibit similar properties: learning, flexibility, parallel search, generalization, and association. This article takes up the problem of the design of a common model for neural networks and adaptive expert systems. For this purpose the Calculus of Self-Modifiable Algorithms, a general tool for problem solving, is used. This joint approach to expert systems and neural networks emphasize their analogies, rather than their differences. © 1993 John Wiley & Sons, Inc.     

智能混合系统研究综述

作为人工智能的两个主要研究方向,专家系统 和神经网络在过去的几十年中均取得了很大的成功,但二者又存在一些各自的缺陷,不能单 独解决一些复杂的工程实际问题．因此,智能混合系统就成为当今人工智能研究领域中的一 个非常重要且具有很大实用价值和广阔发展前景的研究领域．本文分析比较了专家系统和神 经网络的基本特点,论述智能混合系统的研究现状并指出存在的一些问题．     

Neural network models for a resource allocation problem

University admissions and business personnel offices use a limited number of resources to process an ever-increasing quantity of student and employment applications. Application systems are further constrained to identify and acquire, in a limited time period, those candidates who are most likely to accept an offer of enrolment or employment. Neural networks are a new methodology to this particular domain. Various neural network architectures and learning algorithms are analyzed comparatively to determine the applicability of supervised learning neural networks to the domain problem of personnel resource allocation and to identify optimal learning strategies in this domain. This paper focuses on multilayer perceptron backpropagation, radial basis function, counterpropagation, general regression, fuzzy ARTMAP, and linear vector quantization neural networks. Each neural network predicts the probability of enrolment and nonenrolment for individual student applicants. Backpropagation networks produced the best overall performance. Network performance results are measured by the reduction in counsellors student case load and corresponding increases in student enrolment. The backpropagation neural networks achieve a 56% reduction in counsellor case load.     

A simulation study of artificial neural networks for nonlinear time-series forecasting

This study presents an experimental evaluation of neural networks for nonlinear time-series forecasting. The effects of three main factors — input nodes, hidden nodes and sample size, are examined through a simulated computer experiment. Results show that neural networks are valuable tools for modeling and forecasting nonlinear time series while traditional linear methods are not as competent for this task. The number of input nodes is much more important than the number of hidden nodes in neural network model building for forecasting. Moreover, large sample is helpful to ease the overfitting problem.Scope and purposeInterest in using artificial neural networks for forecasting has led to a tremendous surge in research activities in the past decade. Yet, mixed results are often reported in the literature and the effect of key modeling factors on performance has not been thoroughly examined. The lack of systematic approaches to neural network model building is probably the primary cause of inconsistencies in reported findings. In this paper, we present a systematic investigation of the application of neural networks for nonlinear time-series analysis and forecasting. The purpose is to have a detailed examination of the effects of certain important neural network modeling factors on nonlinear time-series modeling and forecasting.     

Comparison of intelligent systems in detecting a child’s mathematical gift

This paper compares the efficiency of two intelligent methods: expert systems and neural networks, in detecting children’s mathematical gift at the fourth grade of elementary school. The input space for the expert system and the neural network model consisted of 60 variables describing five basic components of a child’s mathematical gift identified in previous research. The expert system estimated a child’s gift based on heuristically defined logic rules, while the scientifically confirmed psychological evaluation of gift based on Raven’s standard progressive matrices was used at the output of neural network models. Three neural network algorithms were tested on a Croatian dataset. The results show that both the expert system and the neural network recognize more pupils as mathematically gifted than teachers do. The expert system produces the highest average hit rate, although the highest accuracy in classifying gifted children is obtained by the radial basis neural network algorithm, which also yields lower type II error. Due to the ability of expert systems to explain the result, it can be suggested that both the expert system and the neural network model have potential to serve as effective intelligent decision support tools in detecting mathematical gift in early stage, therefore enabling its further development.     

Neural computing for software reliability*

Abstract: We present the results of a feasibility study for the application of neural computing to the traditional problem of how to generate cost-effective, reliable implementations of complex problems—i.e. the central problem of software engineering. We treat neural computing as an innovative technology for conventional software engineering. We explore the reliability of neural networks (multilayer perceptrons trained with the backpropagation algorithm) as alternative versions in a multiversion software system. The basic idea is that versions trained differently will not exhibit common faults as independently developed, conventional versions (programmed in, for example, Modula-2) have been shown to do. The common design faults that run through independently developed versions appear to be the result of ‘difficult’ inputs which all programmers tend to misconstrue similarly. Network implementations, which are not directly designed in the conventional manner, should permit easy introduction of ‘diversity’ to combat this weakness. The initial results give credence to this possibility and have shown the way to generate substantial forced diversity within the neural computing paradigm.     

A Monte Carlo EM approach for partially observable diffusion processes: theory and applications to neural networks

We present a Monte Carlo approach for training partially observable diffusion processes. We apply the approach to diffusion networks, a stochastic version of continuous recurrent neural networks. The approach is aimed at learning probability distributions of continuous paths, not just expected values. Interestingly, the relevant activation statistics used by the learning rule presented here are inner products in the Hilbert space of square integrable functions. These inner products can be computed using Hebbian operations and do not require backpropagation of error signals. Moreover, standard kernel methods could potentially be applied to compute such inner products. We propose that the main reason that recurrent neural networks have not worked well in engineering applications (e.g., speech recognition) is that they implicitly rely on a very simplistic likelihood model. The diffusion network approach proposed here is much richer and may open new avenues for applications of recurrent neural networks. We present some analysis and simulations to support this view. Very encouraging results were obtained on a visual speech recognition task in which neural networks outperformed hidden Markov models.     

Back-propagation neural networks for modeling complex systems

In complex engineering systems, empirical relationships are often employed to estimate design parameters and engineering properties. A complex domain is characterized by a number of interacting factors and their relationships are, in general, not precisely known. In addition, the data associated with these parameters are usually incomplete or erroneous (noisy). The development of these empirical relationships is a formidable task requiring sophisticated modeling techniques as well as human intuition and experience. This paper demonstrates the use of back-propagation neural networks to alleviate this problem. Backpropagation neural networks are a product of artificial intelligence research. First, an overview of the neural network methodology is presented. This is followed by some practical guidelines for implementing back-propagation neural networks. Two examples are then presented to demonstrate the potential of this approach for capturing nonlinear interactions between variables in complex engineering systems.     

Advanced self-organizing polynomial neural network

In this paper, we introduce a concept of advanced self-organizing polynomial neural network (Adv_SOPNN). The SOPNN is a flexible neural architecture whose structure is developed through a modeling process. But the SOPNN has a fatal drawback; it cannot be constructed for nonlinear systems with few input variables. To relax this limitation of the conventional SOPNN, we combine a fuzzy system and neural networks with the SOPNN. Input variables are partitioned into several subspaces by the fuzzy system or neural network, and these subspaces are utilized as new input variables to the SOPNN architecture. Two types of the advanced SOPNN are obtained by combining not only the fuzzy rules of a fuzzy system with SOPNN but also the nodes in a hidden layer of neural networks with SOPNN into one methodology. The proposed method is applied to the nonlinear system with two inputs, which cannot be identified by conventional SOPNN to show the performance of the advanced SOPNN. The results show that the proposed method is efficient for systems with limited data set and a few input variables and much more accurate than other modeling methods with respect to identification error.