Dynamic analyses of information encoding in neural ensembles

Neural spike train decoding algorithms and techniques to compute Shannon mutual information are important methods for analyzing how neural systems represent biological signals. Decoding algorithms are also one of several strategies being used to design controls for brain-machine interfaces. Developing optimal strategies to design decoding algorithms and compute mutual information are therefore important problems in computational neuroscience. We present a general recursive filter decoding algorithm based on a point process model of individual neuron spiking activity and a linear stochastic state-space model of the biological signal. We derive from the algorithm new instantaneous estimates of the entropy, entropy rate, and the mutual information between the signal and the ensemble spiking activity. We assess the accuracy of the algorithm by computing, along with the decoding error, the true coverage probability of the approximate 0.95 confidence regions for the individual signal estimates. We illustrate the new algorithm by reanalyzing the position and ensemble neural spiking activity of CA1 hippocampal neurons from two rats foraging in an open circular environment. We compare the performance of this algorithm with a linear filter constructed by the widely used reverse correlation method. The median decoding error for Animal 1 (2) during 10 minutes of open foraging was 5.9 (5.5) cm, the median entropy was 6.9 (7.0) bits, the median information was 9.4 (9.4) bits, and the true coverage probability for 0.95 confidence regions was 0.67 (0.75) using 34 (32) neurons. These findings improve significantly on our previous results and suggest an integrated approach to dynamically reading neural codes, measuring their properties, and quantifying the accuracy with which encoded information is extracted.     

基于神经网络集成的专家系统模型

提出一种基于神经网络集成的专家系统模型,并给出神经网络集成的构造算法.在该模型中神经网络集成作为专家系统的一个内嵌模块,用于专家系统的知识获取,克服了传统专家系统在知识获取中的＂瓶颈＂问题.并将该模型用于图书剔旧系统中,初步建成基于神经网络集成的图书剔旧专家系统原型.     

Evolving artificial neural network ensembles

Using a coordinated group of simple solvers to tackle a complex problem is not an entirely new idea. Its root could be traced back hundreds of years ago when ancient Chinese suggested a team approach to problem solving. For a long time, engineers have used the divide-and-conquer strategy to decompose a complex problem into simpler sub-problems and then solve them by a group of solvers. However, knowing the best way to divide a complex problem into simpler ones relies heavily on the available domain knowledge. It is often a manual process by an experienced engineer. There have been few automatic divide-and-conquer methods reported in the literature. Fortunately, evolutionary computation provides some of the interesting avenues to automatic divide-and-conquer methods. An in-depth study of such methods reveals that there is a deep underlying connection between evolutionary computation and ANN ensembles. Ideas in one area can be usefully transferred into another in producing effective algorithms. For example, using speciation to create and maintain diversity had inspired the development of negative correlation learning for ANN ensembles, and an in-depth study of diversity in ensembles. This paper will review some of the recent work in evolutionary approaches to designing ANN ensembles.     

Diagnosis of valvular heart disease through neural networks ensembles

In the last decades, several tools and various methodologies have been proposed by the researchers for developing effective medical decision support systems. Moreover, new methodologies and new tools are continued to develop and represent day by day. Diagnosing of the valvular heart disease is one of the important issue and many researchers investigated to develop intelligent medical decision support systems to improve the ability of the physicians. In this paper, we introduce a methodology which uses SAS Base Software 9.1.3 for diagnosing of the valvular heart disease. A neural networks ensemble method is in the centre of the proposed system. The ensemble-based methods creates new models by combining the posterior probabilities or the predicted values from multiple predecessor models. So, more effective models can be created. We performed experiments with proposed tool. We obtained 97.4% classification accuracy from the experiments made on data set containing 215 samples. We also obtained 100% and 96% sensitivity and specificity values, respectively, in valvular heart disease diagnosis.     

A constructive algorithm for training cooperative neural network ensembles

Presents a constructive algorithm for training cooperative neural-network ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on both accuracy and diversity among individual NNs in an ensemble. In order to maintain accuracy among individual NNs, the number of hidden nodes in individual NNs are also determined by a constructive approach. Incremental training based on negative correlation is used in CNNE to train individual NNs for different numbers of training epochs. The use of negative correlation learning and different training epochs for training individual NNs reflect CNNEs emphasis on diversity among individual NNs in an ensemble. CNNE has been tested extensively on a number of benchmark problems in machine learning and neural networks, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, soybean, and Mackey-Glass time series prediction problems. The experimental results show that CNNE can produce NN ensembles with good generalization ability.     

Effective diagnosis of heart disease through neural networks ensembles

In the last decades, several tools and various methodologies have been proposed by the researchers for developing effective medical decision support systems. Moreover, new methodologies and new tools are continued to develop and represent day by day. Diagnosing of the heart disease is one of the important issue and many researchers investigated to develop intelligent medical decision support systems to improve the ability of the physicians. In this paper, we introduce a methodology which uses SAS base software 9.1.3 for diagnosing of the heart disease. A neural networks ensemble method is in the centre of the proposed system. This ensemble based methods creates new models by combining the posterior probabilities or the predicted values from multiple predecessor models. So, more effective models can be created. We performed experiments with the proposed tool. We obtained 89.01% classification accuracy from the experiments made on the data taken from Cleveland heart disease database. We also obtained 80.95% and 95.91% sensitivity and specificity values, respectively, in heart disease diagnosis.     

基于神经网络集成的软件故障预测及实验分析

软件系统故障预测是软件测试过程中软件可靠性研究的重点之一。利用软件系统测试过程中前期的故障相关信息进行建模,预测后期的软件故障信息,以便于后期测试和验证资源的合理分配。根据软件测试过程中已知的软件故障时间序列,利用非齐次泊松分布过程、神经网络、神经网络集成等方法对其进行建模。通过对三个实例分别建模,其预测平均相对误差G-O模型依次为3.02%、5.88%和6.58%,而神经网络集成模型为0.19%、1.88%和1.455%,实验结果表明神经网络集成模型具有更精确的预测能力。     

Analysing the localisation sites of proteins through neural networks ensembles

Scientists involved in the area of proteomics are currently seeking integrated, customised and validated research solutions to better expedite their work in proteomics analyses and drug discoveries. Some drugs and most of their cell targets are proteins, because proteins dictate biological phenotype. In this context, the automated analysis of protein localisation is more complex than the automated analysis of DNA sequences; nevertheless the benefits to be derived are of same or greater importance. In order to accomplish this target, the right choice of the kind of the methods for these applications, especially when the data set is drastically imbalanced, is very important and crucial. In this paper we investigate the performance of some commonly used classifiers, such as the K nearest neighbours and feed-forward neural networks with and without cross-validation, in a class of imbalanced problems from the bioinformatics domain. Furthermore, we construct ensemble-based schemes using the notion of diversity, and we empirically test their performance on the same problems. The experimental results favour the generation of neural network ensembles as these are able to produce good generalisation ability and significant improvement compared to other single classifier methods.     

Cooperative coevolution of artificial neural network ensembles for pattern classification

This paper presents a cooperative coevolutive approach for designing neural network ensembles. Cooperative coevolution is a recent paradigm in evolutionary computation that allows the effective modeling of cooperative environments. Although theoretically, a single neural network with a sufficient number of neurons in the hidden layer would suffice to solve any problem, in practice many real-world problems are too hard to construct the appropriate network that solve them. In such problems, neural network ensembles are a successful alternative. Nevertheless, the design of neural network ensembles is a complex task. In this paper, we propose a general framework for designing neural network ensembles by means of cooperative coevolution. The proposed model has two main objectives: first, the improvement of the combination of the trained individual networks; second, the cooperative evolution of such networks, encouraging collaboration among them, instead of a separate training of each network. In order to favor the cooperation of the networks, each network is evaluated throughout the evolutionary process using a multiobjective method. For each network, different objectives are defined, considering not only its performance in the given problem, but also its cooperation with the rest of the networks. In addition, a population of ensembles is evolved, improving the combination of networks and obtaining subsets of networks to form ensembles that perform better than the combination of all the evolved networks. The proposed model is applied to ten real-world classification problems of a very different nature from the UCI machine learning repository and proben1 benchmark set. In all of them the performance of the model is better than the performance of standard ensembles in terms of generalization error. Moreover, the size of the obtained ensembles is also smaller.     

Dynamic adaptation of online ensembles for drifting data streams

The success of data stream mining techniques has allowed decision makers to analyze their data in multiple domains, ranging from monitoring network intrusion to financial markets analysis and online sales transactions exploration. Specifically, online ensembles that construct accurate models against drifting data streams have been developed. Recently, there has been a surge in interest in mobile (or so-called pocket) data stream mining, aiming to construct near real-time models for data stream mining applications that run on mobile devices. In such a setting, it follows that the computational resources are limited and that there is a need to adapt analytics to map the resource usage requirements. Consequently, the resultant models should not only be highly accurate, but they should also adapt swiftly to changes. In addition, the data mining techniques should be fast, scalable, and efficient in terms of resource allocation. It then becomes important to consider Return on Investment (ROI) issues such as storage requirements and memory utilization. This paper introduces the Adaptive Ensemble Size (AES) algorithm, an extension of the Online Bagging method, to address these issues. Our AES method dynamically adapts the sizes of ensembles, based on ROI usage patterns. We illustrate our approach by analyzing the performances against both synthetic and real-world data streams. The results, when comparing our AES algorithm with the state-of-the-art, indicate that we are able to obtain a high Return on Investment (ROI) and to swiftly adapt to change, without compromising on the predictive accuracy.     

Spike-frequency adapting neural ensembles: beyond mean adaptation and renewal theories

We propose a Markov process model for spike-frequency adapting neural ensembles that synthesizes existing mean-adaptation approaches, population density methods, and inhomogeneous renewal theory, resulting in a unified and tractable framework that goes beyond renewal and mean-adaptation theories by accounting for correlations between subsequent interspike intervals. A method for efficiently generating inhomogeneous realizations of the proposed Markov process is given, numerical methods for solving the population equation are presented, and an expression for the first-order interspike interval correlation is derived. Further, we show that the full five-dimensional master equation for a conductance-based integrate-and-fire neuron with spike-frequency adaptation and a relative refractory mechanism driven by Poisson spike trains can be reduced to a two-dimensional generalization of the proposed Markov process by an adiabatic elimination of fast variables. For static and dynamic stimulation, negative serial interspike interval correlations and transient population responses, respectively, of Monte Carlo simulations of the full five-dimensional system can be accurately described by the proposed two-dimensional Markov process.     

Dynamic multi-objective evolution of classifier ensembles for video face recognition

Due to a limited control over changing operational conditions and personal physiology, systems used for video-based face recognition are confronted with complex and changing pattern recognition environments. Although a limited amount of reference data is initially available during enrollment, new samples often become available over time, through re-enrollment, post analysis and labeling of operational data, etc. Adaptive multi-classifier systems (AMCSs) are therefore desirable for the design and incremental update of facial models. For real time recognition of individuals appearing in video sequences, facial regions are captured with one or more cameras, and an AMCS must perform fast and efficient matching against the facial model of individual enrolled to the system. In this paper, an incremental learning strategy based on particle swarm optimization (PSO) is proposed to efficiently evolve heterogeneous classifier ensembles in response to new reference data. This strategy is applied to an AMCS where all parameters of a pool of fuzzy ARTMAP (FAM) neural network classifiers (i.e., a swarm of classifiers), each one corresponding to a particle, are co-optimized such that both error rate and network size are minimized. To provide a high level of accuracy over time while minimizing the computational complexity, the AMCS integrates information from multiple diverse classifiers, where learning is guided by an aggregated dynamical niching PSO (ADNPSO) algorithm that optimizes networks according both these objectives. Moreover, pools of FAM networks are evolved to maintain (1) genotype diversity of solutions around local optima in the optimization search space and (2) phenotype diversity in the objective space. Accurate and low cost ensembles are thereby designed by selecting classifiers on the basis of accuracy, and both genotype and phenotype diversity. For proof-of-concept validation, the proposed strategy is compared to AMCSs where incremental learning of FAM networks is guided through mono- and multi-objective optimization. Performance is assessed in terms of video-based error rate and resource requirements under different incremental learning scenarios, where new data is extracted from real-world video streams (IIT-NRC and MoBo). Simulation results indicate that the proposed strategy provides a level of accuracy that is comparable to that of using mono-objective optimization and reference face recognition systems, yet requires a fraction of the computational cost (between 16% and 20% of a mono-objective strategy depending on the data base and scenario).     

Dynamic modelling using neural networks

A neural network structure is presented that uses feedback of unmeasured system states to represent dynamic systems more efficiently than conventional feedforward and recurrent networks, leading to better predictions, reduced training requirement and more reliable extrapolation. The structure identifies the actual system states based on imperfect knowledge of the initial state, which is available in many practical systems, and is therefore applicable only to such systems. It also enables a natural integration of any available partial state-space model directly into the prediction scheme, to achieve further performance improvement. Simulation examples of three varied dynamic systems illustrate the various options and advantages offered by the state-feedback neural structure. Although the advantages of the proposed structure, compared with the conventional feedforward and recurrent networks, should hold for most practical dynamic systems, artificial systems can readily be created and real systems can surely be found for which one or more of these advantages would vanish or even get reversed. Caution is therefore recommended against interpreting the suggested advantages as strict theorems valid in all situations.     

Extrasynaptic-GABA-mediated neuromodulation in a sensory cortical neural network

Simulating a neural network model of an early sensory cortical area, we investigated how gamma-aminobutyric acid (GABA) accumulated in extracellular space (ambient GABA), which depends on the synaptic activity of GABAergic interneurons, acts on the GABAa-receptors located on extrasynaptic membrane regions of principal cells (P), feedback inhibitory cells (F) and lateral inhibitory cells (L). The ambient GABA enhanced the selective responsiveness of P-cells to a target feature stimulus, if it acted on the extrasynaptic GABAa-receptors of P-cells. The ambient GABA led to depolarizing P-cells during ongoing (spontaneous) neuronal-activity periods, if it acted on the extrasynaptic GABAa-receptors of F or L cells. This membrane depolarization contributed to establishing an ongoing subthreshold neuronal state, by which the P-cells could respond quickly to the target stimulus. We suggest that the combinatorial inhibition of P, F, and L cells, meditated by extrasynaptic GABAa-receptors recognizing ambient GABA, is crucial for processing the information of relevant sensory features and for establishing an ongoing subthreshold cortical state that prepares as a ready state for subsequent sensory input. A failure in neuronal-activity-dependent regulation of ambient GABA, stemming largely from the depletion of GABA in extracellular space during senescence, may cause the degeneration of intracortical inhibition that leads to cognitive dysfunction in old animals.     

A canonical neural circuit for cortical nonlinear operations

A few distinct cortical operations have been postulated over the past few years, suggested by experimental data on nonlinear neural response across different areas in the cortex. Among these, the energy model proposes the summation of quadrature pairs following a squaring nonlinearity in order to explain phase invariance of complex V1 cells. The divisive normalization model assumes a gain-controlling, divisive inhibition to explain sigmoid-like response profiles within a pool of neurons. A gaussian-like operation hypothesizes a bell-shaped response tuned to a specific, optimal pattern of activation of the presynaptic inputs. A max-like operation assumes the selection and transmission of the most active response among a set of neural inputs. We propose that these distinct neural operations can be computed by the same canonical circuitry, involving divisive normalization and polynomial nonlinearities, for different parameter values within the circuit. Hence, this canonical circuit may provide a unifying framework for several circuit models, such as the divisive normalization and the energy models. As a case in point, we consider a feedforward hierarchical model of the ventral pathway of the primate visual cortex, which is built on a combination of the gaussian-like and max-like operations. We show that when the two operations are approximated by the circuit proposed here, the model is capable of generating selective and invariant neural responses and performing object recognition, in good agreement with neurophysiological data.     

Dynamic planning model for determining cutting parameters using neural networks in feature-based process planning

Although feature-based computer-aided process planning plays a vital role in automating and integrating design and manufacturing for efficient production, its off-line properties prohibit the shop floor controllers from rapidly coping with unexpected production errors. The objective of the paper is to suggest a neural network-based dynamic planning model, by which the shop floor controllers determine cutting parameters in real-time based on shop floor status. At off-line is the dynamic planning model constructed as a neural network form, and then embedded into each removal feature. The dynamic planning model will be executed by the shop floor controllers to determine the cutting parameters. A prototype system is constructed to validate whether the dynamic planning model is capable of determining dynamically and efficiently the cutting parameters for a particular set of shop operating factors. Owing to the dynamic planning model, the shop floor controller will increase flexibility and robustness by rapidly and adaptively determining the cutting parameters in unexpected errors occurring.     

EEM: evolutionary ensembles model for activity recognition in Smart Homes

Activity recognition requires further research to enable a multitude of human-centric applications in the smart home environment. Currently, the major challenges in activity recognition include the domination of major activities over minor activities, their non-deterministic nature and the lack of availability of human-understandable output. In this paper, we introduce a novel Evolutionary Ensembles Model (EEM) that values both minor and major activities by processing each of them independently. It is based on a Genetic Algorithm (GA) to handle the non-deterministic nature of activities. Our evolutionary ensemble learner generates a human-understandable rule profile to ensure a certain level of confidence for performed activities. To evaluate the EEM, we performed experiments on three different real world datasets. Our experiments show significant improvement of 0.6 % to 0.28 % in the F-measures of recognized activities compared to existing counterparts. It is expected that EEM would be a practical solution for the activity recognition problem due to its understandable output and improved accuracy.     

Dynamic image modelling by neural networks

Sometimes an image can be characterized with the help of a dynamic system in terms of a first-order differential equation, e.g. when a scanner output contains additive noise and/or is degraded due to interaction between the sensing elements (the cameras) or by other phenomena. In this paper, an algorithm for image estimation from the noisy output of a scanner is hypothesized. Determination of the parameters A, B, and the C matrices of a dynamic system leads to the design of an estimator whose input is the output of a scanner with uniform speed. A portion of the overall algorithm using neural networks trained by a gradient descent learning algorithm is discussed in detail. In particular, the determination of A. B, C from states, time derivatives of states and inputs are highlighted. The details of implementation are included. The salient points regarding the development of the complete algorithm are discussed. It is hoped that the results achieved so far will give rise to new techniques for the application of neural networks to image enhancement in particular and to image processing in general.