Multistable learning dynamics in second-order neural networks with time-varying delays

In this paper, we present new results on multistability and attractivity of second-order networks with unsupervised Hebbian-type learning component and time-varying delays. By using the properties of activation functions, we divide state space into invariant sets and establish new criteria of coexistence of equilibrium points which are exponentially stable. The attained results show that second-order synaptic interactions and learning behaviour have an important effect on the multistable convergence of the networks. Finally, numerical simulations will illustrate multistable learning dynamics of second-order networks.     

Neuron-Like Approach to Speech Recognition

In this paper, we present a new approach to speech recognition based on A. Zhdanov’s biomorphic neuron-like networks, which is also known as the autonomous adaptive control (AAC) method. In contrast to artificial neural networks (ANNs), a neuron in the AAC method is itself a self-learning pattern recognition system. We attempt to build a speech recognition system as a construction of such neurons without a program component. If this attempt is successful, then we will be able to simulate the natural principle of speech recognition not only in a program way but also via parallel hardware implementations. We understand the speech recognition problem as one of the speech processes in natural nervous systems that is to be simulated.     

Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control

Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and/or complex properties. In most ER research, the sensory–motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating/blocking/changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed.     

An approach to enlarge learning space coverage for robot learningcontrol

In robot learning control, the learning space for executing general motions of multijoint robot manipulators is quite large. Consequently, for most learning schemes, the learning controllers are used as subordinates to conventional controllers or the learning process needs to be repeated each time a new trajectory is encountered, although learning controllers are considered to be capable of generalization. In this paper, we propose an approach for larger learning space coverage in robot learning control. In this approach, a new structure for learning control is proposed to organize information storage via effective memory management. The proposed structure is motivated by the concept of human motor program and consists mainly of a fuzzy system and a cerebellar model articulation controller (CMAC)-type neural network. The fuzzy system is used for governing a number of sampled motions in a class of motions. The CMAC-type neural network is used to generalize the parameters of the fuzzy system, which are appropriate for the governing of the sampled motions, to deal with the whole class of motions. Under this design, in some sense the qualitative fuzzy rules in the fuzzy system are generalized by the CMAC-type neural network and then a larger learning space can be covered. Therefore, the learning effort is dramatically reduced in dealing with a wide range of robot motions, while the learning process is performed only once. Simulations emulating ball carrying under various conditions are presented to demonstrate the effectiveness of the proposed approach     

A constructive algorithm to synthesize arbitrarily connected feedforward neural networks

In this work we present a constructive algorithm capable of producing arbitrarily connected feedforward neural network architectures for classification problems. Architecture and synaptic weights of the neural network should be defined by the learning procedure. The main purpose is to obtain a parsimonious neural network, in the form of a hybrid and dedicate linear/nonlinear classification model, which can guide to high levels of performance in terms of generalization. Though not being a global optimization algorithm, nor a population-based metaheuristics, the constructive approach has mechanisms to avoid premature convergence, by mixing growing and pruning processes, and also by implementing a relaxation strategy for the learning error. The synaptic weights of the neural networks produced by the constructive mechanism are adjusted by a quasi-Newton method, and the decision to grow or prune the current network is based on a mutual information criterion. A set of benchmark experiments, including artificial and real datasets, indicates that the new proposal presents a favorable performance when compared with alternative approaches in the literature, such as traditional MLP, mixture of heterogeneous experts, cascade correlation networks and an evolutionary programming system, in terms of both classification accuracy and parsimony of the obtained classifier.     

Analysing neurobiological models using communicating automata

Two important issues in computational modelling in cognitive neuroscience are: first, how to formally describe neuronal networks (i.e. biologically plausible models of the central nervous system), and second, how to analyse complex models, in particular, their dynamics and capacity to learn. We make progress towards these goals by presenting a communicating automata perspective on neuronal networks. Specifically, we describe neuronal networks and their biological mechanisms using Data-rich Communicating Automata, which extend classic automata theory with rich data types and communication. We use two case studies to illustrate our approach. In the first case study, we model a number of learning frameworks, which vary in respect of their biological detail, for instance the Backpropagation (BP) and the Generalized Recirculation (GeneRec) learning algorithms. We then used the SPIN model checker to investigate a number of behavioral properties of the neural learning algorithms. SPIN is a well-known model checker for reactive distributed systems, which has been successfully applied to many non-trivial problems. The verification results show that the biologically plausible GeneRec learning is less stable than BP learning. In the second case study, we presented a large scale (cognitive-level) neuronal network, which models an attentional spotlight mechanism in the visual system. A set of properties of this model was verified using Uppaal, a popular real-time model checker. The results show that the asynchronous processing supported by concurrency theory is not only a more biologically plausible way to model neural systems, but also provides a better performance in cognitive modelling of the brain than conventional artificial neural networks that use synchronous updates. Finally, we compared our approach with several other related theories that apply formal methods to cognitive modelling. In addition, the practical implications of the approach are discussed in the context of neuronal network based controllers.     

An Heterogeneous, Endogenous and Coevolutionary GP-Based Financial Market

Stock markets are very important in modern societies and their behavior has serious implications for a wide spectrum of the world's population. Investors, governing bodies, and society as a whole could benefit from better understanding of the behavior of stock markets. The traditional approach to analyzing such systems is the use of analytical models. However, the complexity of financial markets represents a big challenge to the analytical approach. Most analytical models make simplifying assumptions, such as perfect rationality and homogeneous investors, which threaten the validity of their results. This motivates alternative methods.In this paper, we report an artificial financial market and its use in studying the behavior of stock markets. This is an endogenous market, with which we model technical, fundamental, and noise traders. Nevertheless, our primary focus is on the technical traders, which are sophisticated genetic programming based agents that co- evolve (by learning based on their fitness function) by predicting investment opportunities in the market using technical analysis as the main tool. With this endogenous artificial market, we identify the conditions under which the statistical properties of price series in the artificial market resemble some of the properties of real financial markets. By performing a careful exploration of the most important aspects of our simulation model, we determine the way in which the factors of such a model affect the endogenously generated price. Additionally, we model the pressure to beat the market by a behavioral constraint imposed on the agents reflecting the Red Queen principle in evolution. We have demonstrated how evolutionary computation could play a key role in studying stock markets, mainly as a suitable model for economic learning on an agent- based simulation.     

The mean time to express synaptic plasticity in integrate-and-express, stochastic models of synaptic plasticity induction

Stochastic models of synaptic plasticity propose that single synapses perform a directed random walk of fixed step sizes in synaptic strength, thereby embracing the view that the mechanisms of synaptic plasticity constitute a stochastic dynamical system. However, fluctuations in synaptic strength present a formidable challenge to such an approach. We have previously proposed that single synapses must interpose an integration and filtering mechanism between the induction of synaptic plasticity and the expression of synaptic plasticity in order to control fluctuations. We analyze a class of three such mechanisms in the presence of possibly non-Markovian plasticity induction processes, deriving expressions for the mean expression time in these models. One of these filtering mechanisms constitutes a discrete low-pass filter that could be implemented on a small collection of molecules at single synapses, such as CaMKII, and we analyze this discrete filter in some detail. After considering Markov induction processes, we examine our own stochastic model of spike-timing-dependent plasticity, for which the probability density functions of the induction of plasticity steps have previously been derived. We determine the dependence of the mean time to express a plasticity step on pre- and postsynaptic firing rates in this model, and we also consider, numerically, the long-term stability against fluctuations of patterns of neuronal connectivity that typically emerge during neuronal development.     

Statistical mechanics of reward-modulated learning in decision-making networks

The neural substrates of decision making have been intensively studied using experimental and computational approaches. Alternative-choice tasks accompanying reinforcement have often been employed in investigations into decision making. Choice behavior has been empirically found in many experiments to follow Herrnstein's matching law. A number of theoretical studies have been done on explaining the mechanisms responsible for matching behavior. Various learning rules have been proved in these studies to achieve matching behavior as a steady state of learning processes. The models in the studies have consisted of a few parameters. However, a large number of neurons and synapses are expected to participate in decision making in the brain. We investigated learning behavior in simple but large-scale decision-making networks. We considered the covariance learning rule, which has been demonstrated to achieve matching behavior as a steady state (Loewenstein & Seung, 2006 ). We analyzed model behavior in a thermodynamic limit where the number of plastic synapses went to infinity. By means of techniques of the statistical mechanics, we can derive deterministic differential equations in this limit for the order parameters, which allow an exact calculation of the evolution of choice behavior. As a result, we found that matching behavior cannot be a steady state of learning when the fluctuations in input from individual sensory neurons are so large that they affect the net input to value-encoding neurons. This situation naturally arises when the synaptic strength is sufficiently strong and the excitatory input and the inhibitory input to the value-encoding neurons are balanced. The deviation from matching behavior is caused by increasing variance in the input potential due to the diffusion of synaptic efficacies. This effect causes an undermatching phenomenon, which has been often observed in behavioral experiments.     

The significance of organellar proteomics for the nervous system

Organellar proteomics is a useful tool for gaining biological insights about structures in the cell. Here, we discuss the tools used in organellar proteomics and the impact of this technique in understanding nervous system function. We will review insights gained from the proteomes of nervous system-specific organelles such as synaptic vesicles and the postsynaptic density. Moreover, we will show how comparison of proteomes between organelles isolated from the nervous system and from other tissues highlight nervous system-specific functions using the examples of clathrin-coated vesicles and RNA granules.     

Oscillations and spiking pairs: behavior of a neuronal model with STDP learning

In a biologically plausible but computationally simplified integrate-and-fire neuronal population, it is observed that transient synchronized spikes can occur repeatedly. However, groups with different properties exhibit different periods and different patterns of synchrony. We include learning mechanisms in these models. The effects of spike-timing-dependent plasticity have been known to play a distinct role in information processing in the central nervous system for several years. In this letter, neuronal models with dynamical synapses are constructed, and we analyze the effect of STDP on collective network behavior, such as oscillatory activity, weight distribution, and spike timing precision. We comment on how information is encoded by the neuronal signaling, when synchrony groups may appear, and what could contribute to the uncertainty in decision making.     

A computational model of lateralization and asymmetries in cortical maps

While recent experimental work has defined asymmetries and lateralization in left and right cortical maps, the mechanisms underlying these phenomena are currently not established. In order to explore some possible mechanisms in theory, we studied a neural model consisting of paired cerebral hemispheric regions interacting via a simulated corpus callosum. Starting with random synaptic strengths, unsupervised (Hebbian) synaptic modifications led to the emergence of a topographic map in one or both hemispheric regions. Because of uncertainties concerning the nature of hemispheric interactions, both excitatory and inhibitory callosal influences were examined independently. A sharp transition in model behavior was observed depending on callosal strength. For excitatory or weakly inhibitory callosal interactions, complete and symmetric mirror-image maps generally appeared in both hemispheric regions. In contrast, with stronger inhibitory callosal interactions, partial to complete map lateralization tended to occur, and the maps in each hemispheric region often became complementary. Lateralization occurred readily toward the side having a larger cortical region or higher excitability. Asymmetric synaptic plasticity, however, had only a transitory effect on lateralization. These results support the hypotheses that interhemispheric competition occurs, that multiple underlying asymmetries may lead to function lateralization, and that the effects of asymmetric synaptic plasticity may vary depending on whether supervised or unsupervised learning is involved. To our knowledge, this is the first computational model to demonstrate the emergence of topographic map lateralization and asymmetries.     

A study of evolution strategy based cooperative behavior in collective agents

The following paper introduces an evolution strategy on the basis of cooperative behaviors in each group of agents. The evolution strategy helps each agent to be self-defendable and self-maintainable. To determine an optimal group behavior strategy under dynamically varying circumstances, agents in same group cooperate with each other. This proposed method use reinforcement learning, enhanced neural network, and artificial life. In the present paper, we apply two different reward models: reward model 1 and reward model 2. Each reward model is designed as considering the reinforcement or constraint of behaviors. In competition environments of agents, the behavior considered to be advantageous is reinforced as adding reward values. On the contrary, the behavior considered to be disadvantageous is constrained as subtracting the values. And we propose an enhanced neural network to add learning behavior of an artificial organism-level to artificial life simulation. In future, the system models and results described in this paper will be applied to the framework of healthcare systems that consists of biosensors, healthcare devices, and healthcare system.     

The number of synaptic inputs and the synchrony of large, sparse neuronal networks

The prevalence of coherent oscillations in various frequency ranges in the central nervous system raises the question of the mechanisms that synchronize large populations of neurons. We study synchronization in models of large networks of spiking neurons with random sparse connectivity. Synchrony occurs only when the average number of synapses, M, that a cell receives is larger than a critical value, Mc. Below Mc, the system is in an asynchronous state. In the limit of weak coupling, assuming identical neurons, we reduce the model to a system of phase oscillators that are coupled via an effective interaction, gamma. In this framework, we develop an approximate theory for sparse networks of identical neurons to estimate Mc analytically from the Fourier coefficients of gamma. Our approach relies on the assumption that the dynamics of a neuron depend mainly on the number of cells that are presynaptic to it. We apply this theory to compute Mc for a model of inhibitory networks of integrate-and-fire (I&F) neurons as a function of the intrinsic neuronal properties (e.g., the refractory period Tr), the synaptic time constants, and the strength of the external stimulus, Iext. The number Mc is found to be nonmonotonous with the strength of Iext. For Tr = 0, we estimate the minimum value of Mc over all the parameters of the model to be 363.8. Above Mc, the neurons tend to fire in smeared one-cluster states at high firing rates and smeared two-or-more-cluster states at low firing rates. Refractoriness decreases Mc at intermediate and high firing rates. These results are compared to numerical simulations. We show numerically that systems with different sizes, N, behave in the same way provided the connectivity, M, is such that 1/Meff = 1/M - 1/N remains constant when N varies. This allows extrapolating the large N behavior of a network from numerical simulations of networks of relatively small sizes (N = 800 in our case). We find that our theory predicts with remarkable accuracy the value of Mc and the patterns of synchrony above Mc, provided the synaptic coupling is not too large. We also study the strong coupling regime of inhibitory sparse networks. All of our simulations demonstrate that increasing the coupling strength reduces the level of synchrony of the neuronal activity. Above a critical coupling strength, the network activity is asynchronous. We point out a fundamental limitation for the mechanisms of synchrony relying on inhibition alone, if heterogeneities in the intrinsic properties of the neurons and spatial fluctuations in the external input are also taken into account.     

Adaptive WTA With an Analog VLSI Neuromorphic Learning Chip

In this paper, we demonstrate how a particular spike-based learning rule (where exact temporal relations between input and output spikes of a spiking model neuron determine the changes of the synaptic weights) can be tuned to express rate-based classical Hebbian learning behavior (where the average input and output spike rates are sufficient to describe the synaptic changes). This shift in behavior is controlled by the input statistic and by a single time constant. The learning rule has been implemented in a neuromorphic very large scale integration (VLSI) chip as part of a neurally inspired spike signal image processing system. The latter is the result of the European Union research project Convolution AER Vision Architecture for Real-Time (CAVIAR). Since it is implemented as a spike-based learning rule (which is most convenient in the overall spike-based system), even if it is tuned to show rate behavior, no explicit long term average signals are computed on the chip. We show the rule's rate-based Hebbian learning ability in a classification task in both simulation and chip experiment, first with artificial stimuli and then with sensor input from the CAVIAR system     

Neurons with two sites of synaptic integration learn invariant representations.

Neurons in mammalian cerebral cortex combine specific responses with respect to some stimulus features with invariant responses to other stimulus features. For example, in primary visual cortex, complex cells code for orientation of a contour but ignore its position to a certain degree. In higher areas, such as the inferotemporal cortex, translation-invariant, rotation-invariant, and even view point-invariant responses can be observed. Such properties are of obvious interest to artificial systems performing tasks like pattern recognition. It remains to be resolved how such response properties develop in biological systems. Here we present an unsupervised learning rule that addresses this problem. It is based on a neuron model with two sites of synaptic integration, allowing qualitatively different effects of input to basal and apical dendritic trees, respectively. Without supervision, the system learns to extract invariance properties using temporal or spatial continuity of stimuli. Furthermore, top-down information can be smoothly integrated in the same framework. Thus, this model lends a physiological implementation to approaches of unsupervised learning of invariant-response properties.     

Pareto analysis evolutionary and learning systems

This paper attempts to argue that most adaptive systems, such as evolutionary or learning systems, have inherently multiple objectives to deal with. Very often, there is no single solution that can optimize all the objectives. In this case, the concept of Pareto optimality is key to analyzing these systems. To support this argument, we first present an example that considers the robustness and evolvability trade-off in a redundant genetic representation for simulated evolution. It is well known that robustness is critical for biological evolution, since without a sufficient degree of mutational robustness, it is impossible for evolution to create new functionalities. On the other hand, the genetic representation should also provide the chance to find new phenotypes, i.e., the ability to innovate. This example shows quantitatively that a trade-off between robustness and innovation does exist in the studied redundant representation. Interesting results will also be given to show that new insights into learning problems can be gained when the concept of Pareto optimality is applied to machine learning. In the first example, a Pareto-based multi-objective approach is employed to alleviate catastrophic forgetting in neural network learning. We show that learning new information and memorizing learned knowledge are two conflicting objectives, and a major part of both information can be memorized when the multi-objective learning approach is adopted. In the second example, we demonstrate that a Pareto-based approach can address neural network regularizationmore elegantly. By analyzing the Pareto-optimal solutions, it is possible to identifying interesting solutions on the Pareto front.     

eTeacher: Providing personalized assistance to e-learning students

In this paper we present eTeacher, an intelligent agent that provides personalized assistance to e-learning students. eTeacher observes a student’s behavior while he/she is taking online courses and automatically builds the student’s profile. This profile comprises the student’s learning style and information about the student’s performance, such as exercises done, topics studied, exam results. In our approach, a student’s learning style is automatically detected from the student’s actions in an e-learning system using Bayesian networks. Then, eTeacher uses the information contained in the student profile to proactively assist the student by suggesting him/her personalized courses of action that will help him/her during the learning process. eTeacher has been evaluated when assisting System Engineering students and the results obtained thus far are promising.     

Stable reinforcement learning with recurrent neural networks

In this paper,we present a technique for ensuring the stability of a large class of adaptively controlled systems.We combine IQC models of both the controlled system and the controller with a method of filtering control parameter updates to ensure stable behavior of the controlled system under adaptation of the controller.We present a specific application to a system that uses recurrent neural networks adapted via reinforcement learning techniques.The work presented extends earlier works on stable reinforcement learning with neural networks.Specifically,we apply an improved IQC analysis for RNNs with time-varying weights and evaluate the approach on more complex control system.