Learning and classification of monotonic ordinal concepts

Ordinal reasoning plays a major role in human cognition. This paper identifies an important class of classification problems of patterns taken from ordinal domains and presents efficient, incremental algorithms for learning the classification rules from examples. We show that by adopting a monotonicity assumption of the output with respect to the input, inconsistencies among examples can be easily detected and the number of possible classification rules substantially reduced. By adopting a conservative classification criterion, the required number of rules further decreases. The monotonicity and conservatism of the classification also enable the resolution of conflicts among inconsistent examples and the graceful handling of don't knows and don't cares during the learning and classification phases. Two typical examples in which the suggested classification model works well are given. The first example is taken from the financial domain and the second from machining.     

Reorganizing knowledge in neural networks: an explanatory mechanismfor neural networks in data classification problems

We propose an explanatory mechanism for multilayered neural networks (NN). In spite of the effective learning capability as a uniform function approximator, the multilayered NN suffers from unreadability, i.e., it is difficult for the user to interpret or understand the "knowledge" that the NN has by looking at the connection weights and thresholds obtained by backpropagation (BP). This unreadability comes from the distributed nature of the knowledge representation in the NN. In this paper, we propose a method that reorganizes the distributed knowledge in the NN to extract approximate classification rules. Our rule extraction method is based on the analysis of the function that the NN has learned, rather than on the direct interpretation of connection weights as correlation information. More specifically, our method divides the input space into "monotonic regions" where a monotonic region is a set of input patterns that belongs to the same class with the same sensitivity pattern. Approximate classification rules are generated by projecting these monotonic regions.     

Learning neural networks for text classification by exploiting label relations

Multimedia Tools and Applications - Text classification has been a fundamental problem in the realm of Natural Language Processing (NLP), and a variety of approaches have been proposed with the...     

Learning neural networks with noisy inputs using theerrors-in-variables approach

Currently, most learning algorithms for neural-network modeling are based on the output error approach, using a least squares cost function. This method provides good results when the network is trained with noisy output data and known inputs. Special care must be taken, however, when training the network with noisy input data, or when both inputs and outputs contain noise. This paper proposes a novel cost function for learning NN with noisy inputs, based on the errors-in-variables stochastic framework. A learning scheme is presented and examples are given demonstrating the improved performance in neural-network curve fitting, at the cost of increased computation time.     

Improvise approach for respiratory pathologies classification with multilayer convolutional neural networks

Multimedia Tools and Applications - Every respiratory-related checkup includes audio samples collected from the individual, collected through different tools (sonograph, stethoscope). This audio is...     

Markovian architectural bias of recurrent neural networks

In this paper, we elaborate upon the claim that clustering in the recurrent layer of recurrent neural networks (RNNs) reflects meaningful information processing states even prior to training. By concentrating on activation clusters in RNNs, while not throwing away the continuous state space network dynamics, we extract predictive models that we call neural prediction machines (NPMs). When RNNs with sigmoid activation functions are initialized with small weights (a common technique in the RNN community), the clusters of recurrent activations emerging prior to training are indeed meaningful and correspond to Markov prediction contexts. In this case, the extracted NPMs correspond to a class of Markov models, called variable memory length Markov models (VLMMs). In order to appreciate how much information has really been induced during the training, the RNN performance should always be compared with that of VLMMs and NPMs extracted before training as the "" base models. Our arguments are supported by experiments on a chaotic symbolic sequence and a context-free language with a deep recursive structure.     

A new approach to artificial neural networks

A novel approach to artificial neural networks is presented. The philosophy of this approach is based on two aspects: the design of task-specific networks, and a new neuron model with multiple synapses. The synapses' connective strengths are modified through selective and cumulative processes conducted by axo-axonic connections from a feedforward circuit. This new concept was applied to the position control of a planar two-link manipulator exhibiting excellent results on learning capability and generalization when compared with a conventional feedforward network. In the present paper, the example shows only a network developed from a neuronal reflexive circuit with some useful artifices, nevertheless without the intention of covering all possibilities devised.     

A neural networks approach to minority game

The minority game (MG) comes from the so-called “El Farol bar” problem by W.B. Arthur. The underlying idea is competition for limited resources and it can be applied to different fields such as: stock markets, alternative roads between two locations and in general problems in which the players in the “minority” win. Players in this game use a window of the global history for making their decisions, we propose a neural networks approach with learning algorithms in order to determine players strategies. We use three different algorithms to generate the sequence of minority decisions and consider the prediction power of a neural network that uses the Hebbian algorithm. The case of sequences randomly generated is also studied. Research supported by Local Project 2004–2006 (EX 40%) Università di Foggia. A. Sfrecola is a researcher financially supported by Dipartimento di Scienze Economiche, Matematiche e Statistiche, Università degli Studi di Foggia, Foggia, Italy.     

A neural network approach to the classification problem

The task of classifying observations into known groups is a common problem in decision making. A wealth of statistical approaches, commencing with Fisher's linear discriminant function, and including variations to accomodate a variety of modeling assumptions, have been proposed. In addition, nonparametric approaches based on various mathematical programming models have also been proposed as solutions. All of these proposed aolutions have performed well when conditions favorable to the specific model are present. The modeler, therefore, can usually be assured of a good solution to his problem of he chooses a model which fits his situation. In this paper, the performance of a neural network as a classifier is evaluated. It is found that the performance of the neural network is comparable to the best of otheother methods under a wide variety of modeling assumptions. The use of neural networks as classifiers thus relieves the modeler of testing assumptions which would otherwise be critical to the performance of the usual classification techniques.     

A unified approach to building and controlling spiking attractor networks

Extending work in Eliasmith and Anderson (2003), we employ a general framework to construct biologically plausible simulations of the three classes of attractor networks relevant for biological systems: static (point, line, ring, and plane) attractors, cyclic attractors, and chaotic attractors. We discuss these attractors in the context of the neural systems that they have been posited to help explain: eye control, working memory, and head direction; locomotion (specifically swimming); and olfaction, respectively. We then demonstrate how to introduce control into these models. The addition of control shows how attractor networks can be used as subsystems in larger neural systems, demonstrates how a much larger class of networks can be related to attractor networks, and makes it clear how attractor networks can be exploited for various information processing tasks in neurobiological systems.     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

Parameterizing neural networks for disease classification

Neural networks are one option to implement decision support systems for health care applications. In this paper, we identify optimal settings of neural networks for medical diagnoses: The study involves the application of supervised machine learning using an artificial neural network to distinguish between gout and leukaemia patients. With the objective to improve the base accuracy (calculated from the initial set-up of the neural network model), several enhancements are analysed, such as the use of hyperbolic tangent activation function instead of the sigmoid function, the use of two hidden layers instead of one, and transforming the measurements with linear regression to obtain a smoothened data set. Another setting we study is the impact on the accuracy when using a data set of reduced size but with higher data quality. We also discuss the tradeoff between accuracy and runtime efficiency.     

Negatively correlated neural networks for classification

This paper presents a new algorithm for designing neural network ensembles for classification problems with noise. The idea behind this new algorithm is to encourage different individual networks in an ensemble to learn different parts or aspects of the training data so that the whole ensemble can learn the whole training data better. Negatively correlated neural networks are trained with a novel correlation penalty term in the error function to encourage such specialization. In our algorithm, individual networks are trained simultaneously rather than independently or sequentially. This provides an opportunity for different networks to interact with each other and to specialize. Experiments on two real-world problems demonstrate that the new algorithm can produce neural network ensembles with good generalization ability. This work was presented, in part, at the Third International Symposium on Artificial Life and Robotics, Oita, Japan January 19–21, 1998     

Use of probabilistic neural networks to solve classification problems in surgery

The paper considers the classification of peritonitis-stricken patients with regard to the outcome of the operation and the probable result of the treatment. A probabilistic neural network is offered as a classifier.     

Learning in certainty-factor-based multilayer neural networks forclassification

The computational framework of rule-based neural networks inherits from the neural network and the inference engine of an expert system. In one approach, the network activation function is based on the certainty factor (CF) model of MYCIN-like systems. In this paper, it is shown theoretically that the neural network using the CF-based activation function requires relatively small sample sizes for correct generalization. This result is also confirmed by empirical studies in several independent domains.     

Artificial neural networks applied to odor classification for chemical compounds

Three-layer artificial neural networks (ANN) with back-propagation of error were used to classify the odors of chemical compounds. The network's architecture and parameters were optimized, and an empirical rule for dynamically adjusting the learning rate of network was put forward. It was found that the network gave the maximum recognition rate of 100% and converged quickly by using the plural semiconductor gas sensors' response data in combination with the molecular structure codes of odorants. The network's predictive ability for untrained samples was also satisfactory.     

Thalassaemia classification by neural networks and genetic programming

This paper presents the use of a neural network and a decision tree, which is evolved by genetic programming (GP), in thalassaemia classification. The aim is to differentiate between thalassaemic patients, persons with thalassaemia trait and normal subjects by inspecting characteristics of red blood cells, reticulocytes and platelets. A structured representation on genetic algorithms for non-linear function fitting or STROGANOFF is the chosen architecture for genetic programming implementation. For comparison, multilayer perceptrons are explored in classification via a neural network. The classification results indicate that the performance of the GP-based decision tree is approximately equal to that of the multilayer perceptron with one hidden layer. But the multilayer perceptron with two hidden layers, which is proven to have the most suitable architecture among networks with different number of hidden layers, outperforms the GP-based decision tree. Nonetheless, the structure of the decision tree reveals that some input features have no effects on the classification performance. The results confirm that the classification accuracy of the multilayer perceptron with two hidden layers can still be maintained after the removal of the redundant input features. Detailed analysis of the classification errors of the multilayer perceptron with two hidden layers, in which a reduced feature set is used as the network input, is also included. The analysis reveals that the classification ambiguity and misclassification among persons with minor thalassaemia trait and normal subjects is the main cause of classification errors. These results suggest that a combination of a multilayer perceptron with a blood cell analysis may give rise to a guideline/hint for further investigation of thalassaemia classification.     

Colour classification by neural networks in graphic arts

This paper presents a hierarchical modular neural network for colour classification in graphic arts, capable of distinguishing among very similar colour classes. The network performs analysis in a rough to fine fashion, and is able to achieve a high average classification speed and a low classification error. In the rough stage of the analysis, clusters of highly overlapping colour classes are detected. Discrimination between such colour classes is performed in the next stage by using additional colour information from the surroundings of the pixel being classified. Committees of networks make decisions in the next stage. Outputs of members of the committees are adaptively fused through the BADD defuzzification strategy or the discrete Choquet fuzzy integral. The structure of the network is automatically established during the training process. Experimental investigations show the capability of the network to distinguish among very similar colour classes that can occur in multicoloured printed pictures. The classification accuracy obtained is sufficient for the network to be used for inspecting the quality of multicoloured prints.     

采用资格迹的神经网络学习控制算法

强化学习是解决自适应问题的重要方法,被广泛地应用于连续状态下的学习控制,然而存在效率不高和收敛速度较慢的问题.在运用反向传播(back propagation,BP)神经网络基础上,结合资格迹方法提出一种算法,实现了强化学习过程的多步更新.解决了输出层的局部梯度向隐层节点的反向传播问题,从而实现了神经网络隐层权值的快速更新,并提供一个算法描述.提出了一种改进的残差法,在神经网络的训练过程中将各层权值进行线性优化加权,既获得了梯度下降法的学习速度又获得了残差梯度法的收敛性能,将其应用于神经网络隐层的权值更新,改善了值函数的收敛性能.通过一个倒立摆平衡系统仿真实验,对算法进行了验证和分析.结果显示,经过较短时间的学习,本方法能成功地控制倒立摆,显著提高了学习效率.     

Evolution of neural networks for classification and regression

Although Artificial Neural Networks (ANNs) are importantdata mining techniques, the search for the optimal ANN is a challenging task: the ANN should learn the input–output mapping without overfitting the data and training algorithms may get trapped in local minima. The use of Evolutionary Computation (EC) is a promising alternative for ANN optimization. This work presents two hybrid EC/ANN algorithms: the first evolves neural topologies while the latter performs simultaneous optimization of architectures and weights. Sixteen real-world tasks were used to test these strategies. Competitive results were achieved when compared with a heuristic model selection and other Data Mining algorithms.