Diverse neural net solutions to a fault diagnosis problem

The development of a neural net system for fault diagnosis in a marine diesel engine is described. Nets were trained to classify combustion quality on the basis of simulated data. Three different types of data were used: pressure, temperature and combined pressure and temperature. Subsequent to training, three nets were selected and combined by means of a majority voter to form a system which achieved 100% generalisation to the test set. This performance is attributable to a reliance on the software engineering concept of diversity. Following experimental evaluation of methods of creating diverse neural nets solutions, it was concluded that the best results should be obtained when data is taken from two different sensors (e.g. a pressure and a temperature sensor), or where this is not possible, when new data sets are created by subjecting a set of inputs to non-linear transformations. These conclusions have far reaching implications for other neural net applications.     

An incremental neural learning framework and its application to vehicle diagnostics

This paper presents a framework for incremental neural learning (INL) that allows a base neural learning system to incrementally learn new knowledge from only new data without forgetting the existing knowledge. Upon subsequent encounters of new data examples, INL utilizes prior knowledge to direct its incremental learning. A number of critical issues are addressed including when to make the system learn new knowledge, how to learn new knowledge without forgetting existing knowledge, how to perform inference using both the existing and the newly learnt knowledge, and how to detect and deal with aged learnt systems. To validate the proposed INL framework, we use backpropagation (BP) as a base learner and a multi-layer neural network as a base intelligent system. INL has several advantages over existing incremental algorithms: it can be applied to a broad range of neural network systems beyond the BP trained neural networks; it retains the existing neural network structures and weights even during incremental learning; the neural network committees generated by INL do not interact with one another and each sees the same inputs and error signals at the same time; this limited communication makes the INL architecture attractive for parallel implementation. We have applied INL to two vehicle fault diagnostics problems: end-of-line test in auto assembly plants and onboard vehicle misfire detection. These experimental results demonstrate that the INL framework has the capability to successfully perform incremental learning from unbalanced and noisy data. In order to show the general capabilities of INL, we also applied INL to three general machine learning benchmark data sets. The INL systems showed good generalization capabilities in comparison with other well known machine learning algorithms.     

Neural net robot controller: Structure and stability proofs

A multilayer neural net (NN) controller for a general serial-link robot arm is developed. The structure of the NN controller is derived using a filtered error approach. It is argued that standard backpropagation tuning, when used for real-time closed-loop control, can yield unbounded NN weights if: (1) the net can not exactly reconstruct a certain required control function, (2) there are bounded unknown disturbances in the robot dynamics, or (3) the robot arm has more than one link (i.e. nonlinear case). On-line weight tuning algorithms including correction terms to backpropagation, plus an added robustifying signal, guarantee tracking as well as bounded weights. The correction terms involve a second-orderforward-propagated wave in the backprop network.     

Integrating spectral indices with environmental parameters for estimating heavy metal concentrations in rice using a dynamic fuzzy neural-network model

A generalized dynamic fuzzy neural network (GDFNN) was created to estimate heavy metal concentrations in rice by integrating spectral indices and environmental parameters. Hyperspectral data, environmental parameters, and heavy metal content were collected from field experiments with different levels of heavy metal pollution (Cu and Cd). Input variables used in the GDFNN model were derived from 10 variables acquired by gray relational analysis. The assessment models for Cd and Cu concentration employed five and six input variables, respectively. The results showed that the GDFNN for estimating Cu and Cd concentrations in rice performed well at prediction with a compact network structure using the training, validation, and testing sets (for Cu, fuzzy rules=9, R² greater than 0.75, and RMSE less than 2.5; for Cd, fuzzy rules=9, R² greater than 0.75, and RMSE less than 1.0). The final GDFNN model was then compared with a back-propagation (BP) neural network model, adaptive-network-based fuzzy interference systems (ANFIS), and a regression model. The accuracies of GDFNN model prediction were usually slightly better than those of the other three models. This demonstrates that the GDFNN model is more suitable for predicting heavy metal concentrations in rice.     

Neural Network Architecture for Synthesis of the Probabilistic Rule Based Classifiers

We introduce a novel neural network architecture, referred to as the normalizing neural network (NNN), where the propagated signals take the form of finite probability distributions. Appropriately tuned NNN can be applied as the compound voting measure while classifying new cases on the basis of approximate decision reducts extracted from the training data. We provide a general scheme of such a classification process, as well as some theoretical issues concerning the NNN construction. We compare the performance of the appropriately learnt NNNs with the fixed voting measures, for some benchmark data sets.     

Parallel neural net training on the KSR1

In modern day pattern recognition, neural nets are used extensively. General use of a feedforward neural net consists of a training phase followed by a classification phase. Classification of an unknown test vector is very fast and only consists of the propagation of the test vector through the neural net. Training involves an optimization procedure and is very time-consuming since a feasible local minimum is sought in high-dimensional weight space. In this paper we present an analysis of a parallel implementation of the backpropagation training algorithm using conjugate gradient optimization for a three-layered, feedforward neural network, on the KSR1 parallel shared-memory machine. We implement two parallel neural net training versions on the KSR1, one using native code, the other using P4, a library of macros and functions. A speedup model is presented which we use to clarify our experimental results. We identify the general requirements which render the parallel implementation useful, compared to the sequential execution of the same neural net training procedure. We determine the usefulness of a library of functions (such as P4) developed to ease the task of the programmer. Using experimental results we further identify the limits in processor utilization for our parallel training algorithm.     

基于广义计算的时间序列数据挖掘算法

首先介绍了广义计算和时间序列数据挖掘,然后提出一种基于广义计算的时间序列数据挖掘算法。该算法由灰色回归与神经网络结合而成,通过累加生成降低原始数据的波动幅度,以取得较好的拟合效果,再利用神经网络来完成数据的还原过程。该模型在对浙江省可持续发展指标的预测中,取得了满意的效果。     

Robust learning from bites for data mining

Some methods from statistical machine learning and from robust statistics have two drawbacks. Firstly, they are computer-intensive such that they can hardly be used for massive data sets, say with millions of data points. Secondly, robust and non-parametric confidence intervals for the predictions according to the fitted models are often unknown. A simple but general method is proposed to overcome these problems in the context of huge data sets. An implementation of the method is scalable to the memory of the computer and can be distributed on several processors to reduce the computation time. The method offers distribution-free confidence intervals for the median of the predictions. The main focus is on general support vector machines (SVM) based on minimizing regularized risks. As an example, a combination of two methods from modern statistical machine learning, i.e. kernel logistic regression and ε-support vector regression, is used to model a data set from several insurance companies. The approach can also be helpful to fit robust estimators in parametric models for huge data sets.     

Prediction of sulphur dioxide concentration using artificial neural networks

A three-layer neural network model with a hidden recurrent layer is used to predict sulphur dioxide concentration and the predicted values are compared with the measured concentrations at three sites in Delhi. The Levenberg–Marquardt algorithm is used to train the network. The neural network is used to simulate the behaviour of the system. A multivariate regression model is also used for comparison with the results obtained by using the neural network model. The study results indicate that the neural network is able to give better predictions with less residual mean square error than those given by multivariate regression models.     

基于粗糙集的多层容错神经网络故障诊断

针对神经网络故障诊断问题中输入属性维数多和数据量庞大的情况,首先利用粗糙集理论对原始数据进行约简,并按照一定的原则选取多个约简;然后对所得到的多个约简分别构建子神经网络,将多个子网络合成统一的容错网络。结合实例应用取得了令人满意的结果,并为高可靠性设备的故障诊断提供了新的思路。     

An attempt for coloring multichannel MR imaging data

This is an elementary research into assigning color values to voxels of multi-channel magnetic resonance imaging (MRI) volume data. The MRI volume data sets obtained under different scanning conditions are transformed into components by independent component analysis (ICA), which enhances the physical characteristics of the tissue. The transfer functions for generating color values from the independent components are obtained by using a radial basis function network, a kind of neural net, by training the network with sample data chosen from the Visible Human female data set (VHF). The resultant color volume data sets correspond well with the full-color cross-sections of the Visible Human data sets     

Well-quasi-ordering hereditarily finite sets

Hereditarily finite sets can be viewed as digraphs, when one interprets sets as vertices, and the membership relation among sets as the adjacency relation among vertices. We study three digraph containment relation (weak and strong immersion, subdivision) between such membership digraphs and subclasses of them, well-quasi-ordered by these three relations. More specifically, we strengthen and generalize our previous result concerning hereditarily finite well-founded sets. We show that only two conditions of the ones previously considered (slimness, requiring that every membership be necessary, and bounded cardinality) are enough for guaranteeing the well-quasi-ordering property. This is best possible, in the sense that neither of them can be dropped without losing the well-quasi-order property. Our proofs are given in a very general context requiring minimal set-theoretic assumptions, and in which slimness is translated as a graph-theoretic property. This allows us to conclude the well-quasi-ordering of an analogous class of non-well-founded sets, or hypersets.     

Distribution forecasting of high frequency time series

The availability of high frequency data sets in finance has allowed the use of very data intensive techniques using large data sets in forecasting. An algorithm requiring fast k-NN type search has been implemented using AURA, a binary neural network based upon Correlation Matrix Memories. This work has also constructed probability distribution forecasts, the volume of data allowing this to be done in a nonparametric manner. In assistance to standard statistical error measures the implementation of simulations has allowed actual measures of profit to be calculated.     

Validation of artificial neural network techniques in the estimation of nitrogen concentration in rape using canopy hyperspectral reflectance data

A systematic comparison of two types of method for estimating the nitrogen concentration of rape is presented: the traditional statistical method based on linear regression and the emerging computationally powerful technique based on artificial neural networks (ANN). Five optimum bands were selected using stepwise regression. Comparison between the two methods was based primarily on analysis of the statistic parameters. The rms. error for the back-propagation network (BPN) was significantly lower than that for the stepwise regression method, and the T-value was higher for BPN. In particular, for the first-difference of inverse-log spectra (log 1/R)′, T-values performed with a 127.71% success rate using BPN. The results show that the neural network is more robust to training and estimating rape nitrogen concentrations using canopy hyperspectral reflectance data.     

Using single layer networks for discrete, sequential data: An example from Natural Language Processing

Natural Language Processing (NLP) is concerned with processing ordinary, unrestricted text. This work takes a new approach to a traditional NLP task, using neural computing methods. A parser which has been successfully implemented is described. It is a hybrid system, in which neural processors operate within a rule based framework. The neural processing components belong to the class of Generalized Single Layer Networks (GSLN). In general, supervised, feed-forward networks need more than one layer to process data. However, in some cases data can be pre-processed with a non-linear transformation, and then presented in a linearly separable form for subsequent processing by a single layer net. Such networks offer advantages of functional transparency and operational speed. For our parser, the initial stage of processing maps linguistic data onto a higher order representation, which can then be analysed by a single layer network. This transformation is supported by information theoretic analysis. Three different algorithms for the neural component were investigated. Single layer nets can be trained by finding weight adjustments based on (a) factors proportional to the input, as in the Perceptron, (b) factors proportional to the existing weights, and (c) an error minimization method. In our experiments generalization ability varies little; method (b) is used for a prototype parser. This is available via telnet.     

Artificial neural network methods for the prediction of framework crystal structures of zeolites from XRD data

Extracting information about the structures of zeolites and other crystalline materials from X-ray diffraction (XRD) data simply by using statistical methods may provide an impetus for the discovery and identification of unknown materials. In this study, the possibility of using artificial neural network methods for relating framework crystal structures to XRD data reported in literature was investigated. Generalized Regression Neural Networks and Radial Basis Function-Based Neural Networks were utilized in the investigations. The results obtained by neural networks, using fivefold cross validation technique, were compared to the actual values as well as to those determined by multilinear regression. The predictions made by these neural network methods were, in general, more reliable than those performed by regression. The best predictions were achieved for the estimation of the framework densities of zeolites, which provided quite small deviations from the actual values.     

Complexity Results on Learning by Neural Nets

We consider the computational complexity of learning by neural nets. We are interested in how hard it is to design appropriate neural net architectures and to train neural nets for general and specialized learning tasks. Our main result shows that the training problem for 2-cascade neural nets (which have only two non-input nodes, one of which is hidden) is N-complete, which implies that finding an optimal net (in terms of the number of non-input units) that is consistent with a set of examples is also N-complete. This result also demonstrates a surprising gap between the computational complexities of one-node (perceptron) and two-node neural net training problems, since the perceptron training problem can be solved in polynomial time by linear programming techniques. We conjecture that training a k-cascade neural net, which is a classical threshold network training problem, is also N-complete, for each fixed k2. We also show that the problem of finding an optimal perceptron (in terms of the number of non-zero weights) consistent with a set of training examples is N-hard.Our neural net learning model encapsulates the idea of modular neural nets, which is a popular approach to overcoming the scaling problem in training neural nets. We investigate how much easier the training problem becomes if the class of concepts to be learned is known a priori and the net architecture is allowed to be sufficiently non-optimal. Finally, we classify several neural net optimization problems within the polynomial-time hierarchy.     

化学因子分析法测定液体推进剂A

基于测定推进剂Ａ现有分析方法准确度低,神经网络法对仪器要求高等不足,提出了的一种新的分析方法。该方法利用已知酸的电位滴定数据建立多元线性回归数学模型,利用化学因子分解该模型,对未知样进行浓度测定。实际测定结果表明本方法的最大分析误差不超过０．１７％,标准偏差不超过１．９％,满足样品分析要求。     

Using Boolean factors for the construction of an artificial neural networks

ABSTRACT

We propose a novel approach to define Artificial Neural Network(ANN) architecture from Boolean factors. ANNs are a subfield of machine learning applicable to several areas of life. However, defining its architecture for solving a given problem is not formalized and remains an open research problem. Since it is difficult to look into the network and figure out exactly what it has learnt, the complexity of such a technique makes its interpretation more tedious. We propose in this paper to build feedforward ANNs using the optimal factors obtained from the Boolean context representing a data. Since optimal factors completely cover the data and therefore give an explanation to these data, We could give an interpretation to the neurons activation and justify the presence of a neuron in our proposed neural network. We show through experiments and comparisons on the use data sets that this approach provides relatively better results for some key performance measures.