An educational tool for artificial neural networks

Artificial neural networks are some kind of data processing systems, which try to simulate features of the human brain and its learning process. So, they are widely used by researchers to solve different problems in optimization, classification, pattern recognition, associative memory and control. In this paper, an educational tool, which can be used to work on different kinds of neural network models and learn fundamentals of the artificial neural network, is described. At this point, the whole tool environment provides an advanced system to ensure mentioned functions. The developed system supports using MLP, LVQ and SOM models and related learning algorithms. It employs some visual, interactive tools, which enable users to compose their own neural networks and work on the developed networks easily. By using these tools, users can also understand and learn working mechanism of a typical artificial neural network, using features of different models and related learning algorithms.     

Design of artificial neural networks for tool wear monitoring

An on-line scheme for tool wear monitoring using artificial neural networks (ANNs) has been proposed. Cutting velocity, feed, cutting force and machining time are given as inputs to the ANN, and the flank wear is estimated using the ANN. Different ANN structures are designed and investigated to estimate the tool wear accurately. An existing analytical model is used to obtain the data for various cutting conditions in order to eliminate the huge cost and time associated with generation of training and evaluation data. Motivated by the fact that the tool wear at a given instance of time depends on the tool wear value at a previous instance of time, memory is included in the ANN. ANNs without memory, with one-phase memory, and with two-phase memory are investigated in this study. The effect of various training parameters, such as learning coefficient, momentum, temperature, and number of hidden neurons, on these architectures is studied. The findings and experience obtained should facilitate the design and implementation of reliable and economical real-time systems for tool wear monitoring and identification in intelligent manufacturing.     

Evolutionary artificial neural networks: a review

This paper reviews the use of evolutionary algorithms (EAs) to optimize artificial neural networks (ANNs). First, we briefly introduce the basic principles of artificial neural networks and evolutionary algorithms and, by analyzing the advantages and disadvantages of EAs and ANNs, explain the advantages of using EAs to optimize ANNs. We then provide a brief survey on the basic theories and algorithms for optimizing the weights, optimizing the network architecture and optimizing the learning rules, and discuss recent research from these three aspects. Finally, we speculate on new trends in the development of this area.     

COVNET: a cooperative coevolutionary model for evolving artificial neural networks

This paper presents COVNET, a new cooperative coevolutionary model for evolving artificial neural networks. This model is based on the idea of coevolving subnetworks that must cooperate to form a solution for a specific problem, instead of evolving complete networks. The combination of this subnetworks is part of a coevolutionary process. The best combinations of subnetworks must be evolved together with the coevolution of the subnetworks. Several subpopulations of subnetworks coevolve cooperatively and genetically isolated. The individual of every subpopulation are combined to form whole networks. This is a different approach from most current models of evolutionary neural networks which try to develop whole networks. COVNET places as few restrictions as possible over the network structure, allowing the model to reach a wide variety of architectures during the evolution and to be easily extensible to other kind of neural networks. The performance of the model in solving three real problems of classification is compared with a modular network, the adaptive mixture of experts and with the results presented in the bibliography. COVNET has shown better generalization and produced smaller networks than the adaptive mixture of experts and has also achieved results, at least, comparable with the results in the bibliography.     

VLSI technologies for artificial neural networks

VLSI systems, basic integrated circuits, and silicon technologies are discussed. Novel circuit and design principles that provide a foundation for the implementation of a wide variety of neural network models in silicon are described. The key issues for a successful integration of neural systems are identified. The realization of algorithms in silicon is examined. Special-purpose hardware for carrying out the activation and transfer function and for the connection elements is discussed. A brief overview of the current silicon technologies is provided     

ULSI architectures for artificial neural networks

No clear consensus exists about how to exploit the potential for massively parallel implementations of artificial neural networks. Three hardware implementations to demonstrate the key issues surrounding their use are: model specific integrated circuits for neural associative memories, self-organizing feature maps, and local cluster neural networks     

Intelligent-memory architecture for artificial neural networks

Execution of artificial neural networks, especially for online pattern recognition, mainly depends on time-efficient execution of weighted sums. A new architecture achieves this goal, with a computation time superior to the time complexity of sequential von Neumann machines. This architecture uses additional logic to extend the functionality of conventional RAM. The authors discuss an implementation of this architecture that uses reconfigurable logic     

Rule-based artificial neural networks

Neural networks that are integrated with rule-based systems having a knowledge base offer more capabilities than networks not integrated with such systems.     

Knowledge-based artificial neural networks

Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN (Knowledge-Based Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.     

SyReNN: A tool for analyzing deep neural networks

International Journal on Software Tools for Technology Transfer - Deep Neural Networks (DNNs) are rapidly gaining popularity in a variety of important domains. Unfortunately, modern DNNs have been...     

Speaker verification using committee neural networks

Security is a major problem in web based access or remote access to data bases. In the present study, the technique of committee neural networks was developed for speech based speaker verification. Speech data from the designated speaker and several imposters were obtained. Several parameters were extracted in the time and frequency domains, and fed to neural networks. Several neural networks were trained and the five best performing networks were recruited into the committee. The committee decision was based on majority voting of the member networks. The committee opinion was evaluated with further testing data. The committee correctly identified the designated speaker in (50 out of 50) 100% of the cases and rejected imposters in (150 out of 150) 100% of the cases. The committee decision was not unanimous in majority of the cases tested.     

Evolving transfer functions for artificial neural networks

The paper describes a methodology for constructing transfer functions for the hidden layer of a back-propagation network, which is based on evolutionary programming. The method allows the construction of almost any mathematical form. It is tested using four benchmark classification problems from the well-known machine intelligence problems repository maintained by the University of California, Irvine. It was found that functions other than the commonly used sigmoidal function could perform well when used as hidden layer transfer functions. Three of the four problems showed improved test results when these evolved functions were used.     

Safety-critical neural computing: explanation and verification in knowledge augmented neural networks

This paper addresses the problem that conventional neural networks cannot contain a priori knowledge and cannot explain their output. A mathematical theory of ‘black-box classifiers’ is developed which covers most of the best known neural architectures. The limitations of the non-model based computational paradigm are discussed; these include inability to predict the behaviour of systems with multiple-valued, discontinuous, catastrophic, and chaotic state spaces. Worse, they include the inability to detect the presence of such systems, when they are working outside their range of competence, and when they are working with data of quality outside their range of experience. Of themselves, neural networks cannot communicate with human decision makers in human terms, often the choice is ‘take it or leave it’. Knowledge-based computation does not necessarily have these drawbacks, and can therefore augment the powerful neural computing paradigm where it is weakest. We consider three fundamental ways of combining the two computational paradigms, and show how the explanation facility of knowledge-based systems can be used to induce explanation on the output of neural subsystems. We conclude with an architecture which we believe to be generic for safety-critical neural computation.     

Distributed memory systems for simulating artificial neural networks

In executing tasks involving intelligent information processing, the human brain performs better than the digital computer. The human brain derives its power from a large number [O(10¹¹)] of neurons which are interconnected by a dense interconnection network [O(10⁵) connections per neuron]. Artificial neural network (ANN) paradigms adopt the structure of the brain to try to emulate the intelligent information processing methods of the brain. ANN techniques are being employed to solve problems in areas such as pattern recognition, and robotic processing. Simulation of ANNs involves implementation of large number of neurons and a massive interconnection network. In this paper, we discuss various simulation models of ANNs and their implementation on distributed memory systems. Our investigations reveal that communication-efficient networks of distributed memory systems perform better than other topologies in implementing ANNs.     

Developing artificial neural networks for safety critical systems

There are many performance based techniques that aim to improve the safety of neural networks for safety critical applications. However, many of these approaches provide inadequate forms of safety assurance required for certification. As a result, neural networks are typically restricted to advisory roles in safety-related applications. Neural networks have the ability to operate in unpredictable and changing environments. It is therefore desirable to certify them for highly-dependable roles in safety critical systems. This paper outlines the safety criteria which are safety requirements for the behaviour of neural networks. If enforced, the criteria can contribute to justifying the safety of ANN functional properties. Characteristics of potential neural network models are also outlined and are based upon representing knowledge in interpretable and understandable forms. The paper also presents a safety lifecycle for artificial neural networks. This lifecycle focuses on managing behaviour represented by neural networks and contributes to providing acceptable forms of safety assurance.     

Use of artificial neural networks for clinical diagnosis

The application of two unsupervised Artificial Neural Networks (ART2 and Kohonen) for the classification of patients in three medical fields is considered. In the first case, data obtained from spectral analysis of the EEG signals are used in order to detect brain disfunction of persons exposed to organic solvents. The Contingent Negative variation which is an Evoked Response is utilized in the last two cases for the diagnosis of Huntingdon's Disease and for the separation of schizophrenic subtypes.     

On global-local artificial neural networks for function approximation

We present a hybrid radial basis function (RBF) sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. The algorithm used is intended to identify global features of an input-output relationship before adding local detail to the approximating function. It aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space. We test the effectiveness of our method using five regression tasks; four use synthetic datasets while the last problem uses real-world data on the wave overtopping of seawalls. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization. Our global-local artificial neural network (GL-ANN) is also seen to compare favorably with both perceptron radial basis net and regression tree derived RBFs. A number of issues concerning the training of GL-ANNs are discussed: the use of regularization, the inclusion of a gradient descent optimization step, the choice of RBF spreads, model selection, and the development of appropriate stopping criteria.     

PREPARE: a tool for knowledge base verification

The knowledge base is the most important component in a knowledge-based system. Because a knowledge base is often built in an incremental, piecemeal fashion, potential errors may be inadvertently brought into it. One of the critical issues in developing reliable knowledge-based systems is how to verify the correctness of a knowledge base. The paper describes an automated tool called PREPARE for detecting potential errors in a knowledge base. PREPARE is based on modeling a knowledge base by using a predicate/transition net representation. Inconsistent, redundant, subsumed, circular, and incomplete rules in a knowledge base are then defined as patterns of the predicate/transition net model, and are detected through a syntactic pattern recognition method. The research results to date have indicated that: the methodology ran be adopted in knowledge-based systems where logic is used as knowledge representation formalism; the tool can be invoked at any stage of the system's development, even without a fully functioning inference engine; the predicate/transition net model of knowledge bases is easy to implement and provides a clear and understandable display of the knowledge to be used by the system     

Compartmental modelling with artificial neural networks

In this paper, we introduce an abbreviated compartmental modelling scheme which may be of interest to those in neuron- based adaptive systems because of the additional scope it provides for studying biologically-inspired learning mechanisms. The scheme, although not as flexible and precise as the general compartmental approach, allows one to design Hodgkin-Huxley style cells, and passive dendritic trees with an arbitrary number of synaptic connections. The trade-offs made for computational performance, may make the modelling scheme more appropriate for practical applications. The modelling scheme is based upon artificial neural networks, which we have used to represent cylindrical compartments (both passive and active) of different lengths, two types of voltage-dependent channels, and basic chemical synapses with variable time constants.