Face recognition using artificial neural network group-basedadaptive tolerance (GAT) trees

Recent artificial neural network research has focused on simple models, but such models have not been very successful in describing complex systems (such as face recognition). This paper introduces the artificial neural network group-based adaptive tolerance (GAT) tree model for translation-invariant face recognition, suitable for use in an airport security system. GAT trees use a two-stage divide-and-conquer tree-type approach. The first stage determines general properties of the input, such as whether the facial image contains glasses or a beard. The second stage identifies the individual. Face perception classification, detection of front faces with glasses and/or beards, and face recognition results using GAT trees under laboratory conditions are presented. We conclude that the neural network group-based model offers significant improvement over conventional neural network trees for this task.     

Fault diagnostic of induction motors for equipment reliability and health maintenance based upon Fourier and wavelet analysis

The motor is the workhorse of industry. The issues of preventive and condition-based maintenance, on-line monitoring, system fault detection, diagnosis, and prognosis are of increasing importance. This paper introduces fault detection for induction motors. Stator currents are measured by current meters and stored by time domain. The time domain is not suitable for representing current signals, so the frequency domain is applied to display signals. The Fourier transform is employed to convert signals. After signal conversion, signal features must be extracted by signal processing such as wavelet and spectrum analysis. Features are entered in a pattern classification model such as a neural network model, a polynomial neural network, or a fuzzy inference model. This paper describes fault detection results that use Fourier and wavelet analysis. This combined approach is very useful and powerful for detection signal features.This work was presented in part at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004This work has been supported by “Research Center for Future Logistics Information Technology” hosted by the Ministry of Education in Korea.     

Optimization free neural network approach for solving ordinary and partial differential equations

Current work introduces a fast converging neural network-based approach for solution of ordinary and partial differential equations. Proposed technique eliminates the need of time-consuming optimization procedure for training of neural network. Rather, it uses the extreme learning machine algorithm for calculating the neural network parameters so as to make it satisfy the differential equation and associated boundary conditions. Various ordinary and partial differential equations are treated using this technique, and accuracy and convergence aspects of the procedure are discussed.

     

Discrete adaptive neural network disturbance feedforward compensation for non-linear disturbances in servo-control applications

This article introduces a novel adaptive neural network compensator for feedforward compensation of external disturbances affecting a closed-loop system. The neural network scheme is posed so that a non-linear disturbance model estimate for a measurable disturbance can be adapted for rejection of the disturbance affecting a closed-loop system. The non-linear neural network approach has been particularly developed for ‘mobile’ applications where the adaptation algorithm has to remain simple. For that reason, the theoretical framework justifies a very simple least-mean-square approach suggested in a mobile hard disk drive context. This approach is generalised to a non-linear adaptive neural network (NN) compensation scheme. In addition, usual assumptions are relaxed, so that it is sufficient to model the disturbance model as a stable non-linear system avoiding strictly positive real assumptions. The output of the estimated disturbance model is assumed to be matched to the compensation signal for effectiveness, although for stability this is not necessary. Practical and simulation examples show different features of the adaptation algorithm. In a realistic hard disk drive simulation and a practical application, it is shown that a non-linear adaptive compensation scheme is required for non-linear disturbance compensation providing better performance at similar computational effort in comparison to well-established schemes.     

一种与神经元网络杂交的决策树算法

神经元网络在多数情况下获得的精度要比决策树和回归算法精度高,这是因为它能适应更复杂的模型,同时由于决策树通常每次只使用一个变量来分支,它所对应的识别空间只能是超矩形,这也就比神经元网络简单,粗度不能与神经元网络相比,然而神经元网络需要相对多的学习时间,并且其模型的可理解性不如决策树、Naive-Bayes等方法直观,本文在进行两种算法对复杂模型的识别对比后,提出了一个新的算法NNTree,这是一个决策树和神经元网络杂交的算法,决策树节点包含单变量的分支就象正常的决策树,但是叶子节点包含神经元网络分类器,这个方法针对决策树处理大型数据的效能,保留了决策树的可理解性,改善了神经元网络的学习性能,同时可使这个分类器的精度大大超过这两种算法,尤其在测试更大的数据集复杂模型时更为明显。     

Internal reinforcement in a connectionist genetic programming approach

Genetic programming (GP) can learn complex concepts by searching for the target concept through evolution of a population of candidate hypothesis programs. However, unlike some learning techniques, such as Artificial Neural Networks (ANNs), GP does not have a principled procedure for changing parts of a learned structure based on that structure's performance on the training data. GP is missing a clear, locally optimal update procedure, the equivalent of gradient-descent backpropagation for ANNs. This article introduces a new algorithm, “internal reinforcement”, for defining and using performance feedback on program evolution. This internal reinforcement principled mechanism is developed within a new connectionist representation for evolving parameterized programs, namely “neural programming”. We present the algorithms for the generation of credit and blame assignment in the process of learning programs using neural programming and internal reinforcement. The article includes a comprehensive overview of genetic programming and empirical experiments that demonstrate the increased learning rate obtained by using our principled program evolution approach.     

A Modified Backpropagation Learning Algorithm With Added Emotional Coefficients

Much of the research work into artificial intelligence (AI) has been focusing on exploring various potential applications of intelligent systems with successful results in most cases. In our attempts to model human intelligence by mimicking the brain structure and function, we overlook an important aspect in human learning and decision making: the emotional factor. While it currently sounds impossible to have “machines with emotions,” it is quite conceivable to artificially simulate some emotions in machine learning. This paper presents a modified backpropagation (BP) learning algorithm, namely, the emotional backpropagation (EmBP) learning algorithm. The new algorithm has additional emotional weights that are updated using two additional emotional parameters: anxiety and confidence. The proposed “emotional” neural network will be implemented to a facial recognition problem, and the results will be compared to a similar application using a conventional neural network. Experimental results show that the addition of the two novel emotional parameters improves the performance of the neural network yielding higher recognition rates and faster recognition time.      

Neurocomputing strategies in structural design — decomposition based optimization

The present paper introduces a scheme utilizing neurocomputing strategies for a decomposition approach to large scale optimization problems. In this scheme the modelling capabilities of a backpropagation neural network are employed to detect weak couplings in a system and to effectively decompose it into smaller, more tractable subsystems. When such partitioning of a design space is possible (decomposable systems), independent optimization in each subsystem is performed with a penalty term added to an objective function to eliminate constraint violations in all other subsystems. Dependencies among subsystems are represented in terms of global design variables, and since only partial information is needed, a neural network is used to map relations between global variables and all system constraints. A featuresensitive network (a variant of ahierarchical vector quantization technique, referred to as the HVQ network) is used for this purpose as it offers easy training, approximations of an arbitrary accuracy, and processing of incomplete input vectors. The approach is illustrated with applications to minimum weight sizing of truss structures with multiple design constraints.     

基于神经网络的分类决策树构造

目前基于符号处理的方法是解决分类规则提取问题的主要方法,而基于神经网络的连接主义方法则用的不多,其主要原因在于虽然神经网络的分类精度高,但难于提取其所隐含的分类规则与知识.针对这个问题,结合神经网络的具体特点,该文提出了一种基于神经网络的构造分类决策树的新方法.该方法通过神经网络训练建立各属性与分类结果之间的关系,进而通过提取各属性与分类结果之间的导数关系来建立分类决策树.给出了具体的决策树构造算法.同时为了提高神经网络所隐含关系的提取效果,提出了关系强化约束的概念并建立了具体的模型.实际应用结果证明了算法的有效性.     

A machine learning method for generation of a neural networkarchitecture: a continuous ID3 algorithm

The relation between the decision trees generated by a machine learning algorithm and the hidden layers of a neural network is described. A continuous ID3 algorithm is proposed that converts decision trees into hidden layers. The algorithm allows self-generation of a feedforward neural network architecture. In addition, it allows interpretation of the knowledge embedded in the generated connections and weights. A fast simulated annealing strategy, known as Cauchy training, is incorporated into the algorithm to escape from local minima. The performance of the algorithm is analyzed on spiral data.     

Extracting decision trees from trained neural networks

In this paper we present a methodology for extracting decision trees from input data generated from trained neural networks instead of doing it directly from the data. A genetic algorithm is used to query the trained network and extract prototypes. A prototype selection mechanism is then used to select a subset of the prototypes. Finally, a standard induction method like ID3 or C5.0 is used to extract the decision tree. The extracted decision trees can be used to understand the working of the neural network besides performing classification. This method is able to extract different decision trees of high accuracy and comprehensibility from the trained neural network.     

Symbolic representation of neural networks

Neural networks often surpass decision trees in predicting pattern classifications, but their predictions cannot be explained. This algorithm's symbolic representations make each prediction explicit and understandable. Our approach to understanding a neural network uses symbolic rules to represent the network decision process. The algorithm, NeuroRule, extracts these rules from a neural network. The network can be interpreted by the rules which, in general, preserve network accuracy and explain the prediction process. We based NeuroRule on a standard three layer feed forward network. NeuroRule consists of four phases. First, it builds a weight decay backpropagation network so that weights reflect the importance of the network's connections. Second, it prunes the network to remove irrelevant connections and units while maintaining the network's predictive accuracy. Third, it discretizes the hidden unit activation values by clustering. Finally, it extracts rules from the network with discretized hidden unit activation values     

Improvements in the predictive capability of neural networks

Neural networks can be used to develop effective models of nonlinear systems. Their main advantage being that they can model the vast majority of nonlinear systems to any arbitrary degree of accuracy. The ability of a neural network to predict the behavior of a nonlinear system accurately ought to be improved if there was some mechanism that allows the incorporation of first-principles model information into their training. This study proposes to use information obtained from a first-principle model to impart a sense of “direction” to the neural network model estimate. This is accomplished by modifying the objective function so as to include an additional term that is the difference between the time derivative of the outputs, as predicted by the neural network, and that of the outputs of the first-principles model during the training phase. The performance of a feedforward neural network model that uses this modified objective function is demonstrated on a chaotic process and compared to the conventional feedforward network trained on the usual objective function.     

Power quality disturbance detection and classification using wavelet and RBFNN

This paper introduces a novel approach to detect and classify power quality disturbance in the power system using radial basis function neural network (RBFNN). The proposed method requires less number of features as compared to conventional approach for the identification. The feature extracted through the wavelet is trained by a radial basis function neural network for the classification of events. After training the neural network, the weight obtained is used to classify the Power Quality (PQ) problems. For the classification, 20 types of disturbances are taken into account. The classification performance of RBFNN is compared with feed forward multilayer network (FFML), learning vector quantization (LVQ), probabilistic neural network (PNN) and generalized regressive neural network (GRNN). The classification accuracy of the RBFNN network is improved, just by rewriting the weights and updating the weights with the help of cognitive as well as the social behavior of particles along with fitness value. The simulation results possess significant improvement over existing methods in signal detection and classification.     

Articulatory feature recognition using dynamic Bayesian networks

We describe a dynamic Bayesian network for articulatory feature recognition. The model is intended to be a component of a speech recognizer that avoids the problems of conventional “beads-on-a-string” phoneme-based models. We demonstrate that the model gives superior recognition of articulatory features from the speech signal compared with a state-of-the-art neural network system. We also introduce a training algorithm that offers two major advances: it does not require time-aligned feature labels and it allows the model to learn a set of asynchronous feature changes in a data-driven manner.     

Recurrent neural network approach for cyclic job shop scheduling problem

While cyclic scheduling is involved in numerous real-world applications, solving the derived problem is still of exponential complexity. This paper focuses specifically on modelling the manufacturing application as a cyclic job shop problem and we have developed an efficient neural network approach to minimise the cycle time of a schedule. Our approach introduces an interesting model for a manufacturing production, and it is also very efficient, adaptive and flexible enough to work with other techniques. Experimental results validated the approach and confirmed our hypotheses about the system model and the efficiency of neural networks for such a class of problems.     

Autonomous generation of reflexion-based robot controller using inductive learning

The paper proposes a novel architecture for autonomously generating and managing a robot control system, aiming for the application to planetary rovers which will move in a partially unknown, unstructured environment. The proposed architecture is similar to the well known subsumption architecture in that the movements are governed by a network of various reflexion patterns. The major departures are that firstly it utilizes inductive learning to automatically generate and modify a control architecture, which is, if human is to do, quite a difficult and time consuming task, secondly it employs the concept of “goal sensor” to deal with the system goal more explicitly, and thirdly it compiles the planning results into a reflexion network and decision trees to maintain the strong features of reflexion based planner such as real-timeness, robustness and extensibility. The architecture has been applied to movement control of a certain rover in computer simulations and simple experiments, in which its effectiveness and characteristics have been cleared.     

“Chaotic relaxation” in concurrently asynchronous neural networks

In this paper we analyze a fundamental issue which directly impacts the scalability of current theoretical neural network models to applicative embodiments, in both software as well as hardware. This pertains to the inherent and unavoidable concurrent asynchronicity of emerging fine-grained computational ensembles and the consequent chaotic manifestations in the absence of proper conditioning. The latter concern is particularly significant since the computational inertia of neural networks in general and our dynamical learning formalisms manifests itself substantially, only in massively parallel hardward—optical, VLSI or opto-electronic. We introduce a mathematical framework for systematically reconditioning additive-type models and derive a neuro-operator, based on the chaotic relaxation paradigm whose resulting dynamics are neither “concurrently” synchronous nor “sequentially” asynchronous. Necessary and sufficient conditions guaranteeing concurrent asynchronous convergence are established in terms of contracting operators. Lyapunov exponents are also computed to characterize the network dynamics and to ensure that throughput-limiting “emergent computational chaos” behavior in models reconditioned with concurrently asynchronous algorithms was eliminated.     

基于神经网络的汇率预测

神经网络在理论上具有无限的函数逼近能力。它在预测领域可以取得很好的效果。利用神经网络的数值逼近与记忆功能，根据汇率历史观测数值，可以识别出汇率序列的内在模式。本文首先说明了利用神经网络进行汇率预测的原理和方法。然后着重探讨了神经网络汇率预测的重要步骤。最后根据不同的衡量指标对测试结果进行了分析。     

Artificial neural networks in process estimation and control

In this contribution, the suitability of the artificial neural network methodology for solving some process engineering problems is discussed. First the concepts involved in the formulation of artificial neural networks are presented. Next the suitability of the technique to provide estimates of difficult to measure quality variables is demonstrated by application to industrial data. Measurements from established instruments are used as secondary variables for estimation of the “primary” quality variables. The advantage of using these estimates for feedback control is then demonstrated. The possibility of using neural network models directly within a model-based predictive control strategy is also considered, making use of an on-line optimization routine to determine the future inputs that will minimize the deviations between the desired and predicted outputs. Control is implemented in a receding horizon fashion. Application of the predictive controller to a nonlinear distillation system is used to indicate the potential of the neural network based control philosophy.