Learning sequences of sparse correlated patterns using small-world attractor neural networks: An application to traffic videos

The goal of this work is to learn and retrieve a sequence of highly correlated patterns using a Hopfield-type of attractor neural network (ANN) with a small-world connectivity distribution. For this model, we propose a weight learning heuristic which combines the pseudo-inverse approach with a row-shifting schema. The influence of the ratio of random connectivity on retrieval quality and learning time has been studied. Our approach has been successfully tested on a complex pattern, as it is the case of traffic video sequences, for different combinations of the involved parameters. Moreover, it has demonstrated to be robust with respect to highly variable frame activity.     

Inhibitory synapses in neural networks with sigmoidalnonlinearities

We study the behavior of anti-Hebbian synapses at a neural node that contains the standard sigmoidal nonlinearity. The criterion generalizes the idea already discussed by one of the authors for linear networks and consists in removing the correlation between the input to the synapse, and the node output. We show how the solution, just as for the linear case, is unique and can be learned with a standard anti-Hebbian rule. We suggest how these synapses can be embedded in fully self-organizing networks to generate orthogonal nonlinear components and be used for multidimensional approximation.     

Fast consensus in networks of bounded degree

Summary TheDistributed Consensus problem involvesn processors each of which holds an initial binary vlaue. At mostt of the processors may be faulty and ignore any protocol (even behaving maliciously), yet it is required that the non-faulty processors eventually agree on a value that was initially held by one of them. In this paper we focus on consensus in networks whose degree is bounded, following the work of Dwork, Peleg, Pippenger and Upfal [8]. In such a context, complete consensus among all the correct processors is not possible and some exceptions must be allowed. We first show how to achieve consensus in the butterfly network usingO(t+lognloglogn) one-bit parallel transmission steps, while tolerating the asymptotically optimal number of faulty processors (O(n/logn)) and having the asymptotically minimal number of exceptions (O(tlogt)). This result considerably improves on the running time of existing butterfly consensus protocols [2, 8]. In particular, it replaces the running time ofO(nlognloglogn) of [2] with an asymptotically optimal one. As in [8], we can then decrease the number of exceptions toO(t) by using additional links, while maintaining the same running time. The protocol is derived from a consensus protocol for completely connected networks that is interesting in its own right: it achieves Distributed Consensus with optimal number of processors, asymptotically optimal total bit transfer and nearly optimal number of rounds. Piotr Berman was born in 1955 in Warsaw, Poland, where here progressed from day-care to the degree of Master of Mathematics obtained from the University of Warsaw in 1978. He later studied at the Polish Academy of Sciences and MIT. He received a Ph.D. in Mathematics from MIT in 1985. From 1982 he has been teaching at Penn State, where currently he has a permanent position. His research interests are in two areas: fault-tolerant distributed computing and approximation algorithms. His non-professional hobbies include ancient history and mountain hiking. Juan Alberto Garay is originally from Rosario, Argentina, where he received his degree of Electrical Engineering from the Universidad Nacional de Rosario in 1976, at the age of 21. He then received his Master's degree in Electronic Engineering from the Eindhoven International Institute of the Eindhoven University of Technology (Eindhoven, Holland) in 1981, and his Ph.D. in Computer Science at Penn State University (University Park, PA) in 1989. Between his first and second degrees he worked as a digital design engineer for SOMISA (San Nicola's, Argentina), and between his second and Ph.D. degrees as a systems engineer for IBM Argentina. During the 1989/1990 academic year he was a visiting Assistant Professor at Bucknell University (Lewisburg, PA), and since 1990 he is with IBM's T.J. Watson Research Center (Yorktown Heights, NY). In 1992 he spent 6 months at The Weizmann Institute of Science (Rehovot, Israel) as a postdoctoral fellow. His professional interests include algorithms and lower bounds, distributed computation and fault tolerance. Dr. Garay enjoys tennis, music and philosophy.Preliminary version appeared in Proc 4th International Workshop on Distributed Algorithms, LNCS 486 (Springer-Verlag), pp 321–333, 1990     

Shortest paths in stochastic networks with correlated link costs

The objective is to minimize expected travel time from any origin to a specific destination in a congestible network with correlated link costs. Each link is assumed to be in one of two possible conditions. Conditional probability density functions for link travel times are assumed known for each condition. Conditions over the traversed links are taken into account for determining the optimal routing strategy for the remaining trip. This problem is treated as a multistage adaptive feedback control process. Each stage is described by the physical state (the location of the current decision point) and the information state (the service level of the previously traversed links). Proof of existence and uniqueness of the solution to the basic dynamic programming equations and a solution procedure are provided.     

Input-driven oscillations in networks with excitatory and inhibitory neurons with dynamic synapses

Previous work has shown that networks of neurons with two coupled layers of excitatory and inhibitory neurons can reveal oscillatory activity. For example, B?rgers and Kopell (2003) have shown that oscillations occur when the excitatory neurons receive a sufficiently large input. A constant drive to the excitatory neurons is sufficient for oscillatory activity. Other studies (Doiron, Chacron, Maler, Longtin, & Bastian, 2003; Doiron, Lindner, Longtin, Maler, & Bastian, 2004) have shown that networks of neurons with two coupled layers of excitatory and inhibitory neurons reveal oscillatory activity only if the excitatory neurons receive correlated input, regardless of the amount of excitatory input. In this study, we show that these apparently contradictory results can be explained by the behavior of a single model operating in different regimes of parameter space. Moreover, we show that adding dynamic synapses in the inhibitory feedback loop provides a robust network behavior over a broad range of stimulus intensities, contrary to that of previous models. A remarkable property of the introduction of dynamic synapses is that the activity of the network reveals synchronized oscillatory components in the case of correlated input, but also reflects the temporal behavior of the input signal to the excitatory neurons. This allows the network to encode both the temporal characteristics of the input and the presence of spatial correlations in the input simultaneously.     

Stabilization of linear systems over networks with bounded packet loss

This paper is concerned with the stabilization problem of networked control systems where the main focus is the packet-loss issue. Two types of packet-loss processes are considered. One is the arbitrary packet-loss process, the other is the Markovian packet-loss process. The stability conditions of networked control systems with both arbitrary and Markovian packet losses are established via a packet-loss dependent Lyapunov approach. The corresponding stabilizing controller design techniques are also given based upon the stability conditions. These results are also extended to the unit time delay case. Finally, the numerical example and simulations have demonstrated the usefulness of the developed theory.     

Impact of bounded delay on throughput in multi-hop networks

This paper presents a closed form solution relating to the impact of bounded delays on throughput in multi-hop networks. In contrast to contemporary literature that largely focuses on average delay to estimate the Quality of Service, our model focuses on an upper bound of delay, referred to as delay threshold in this paper. Traffic that exceeds the delay threshold is treated as lost throughput. The results obtained can be used in scaling resources in a multi-hop network for attaining specified levels of throughput under different thresholds of acceptable delays. Both single-hop and multi-hop transfers are addressed. The theoretical analysis presented in this paper is further corroborated by simulation. The findings presented in this paper will be very relevant to multi-hop network applications where received data that are older than a specified threshold period are not relevant and must be discarded.     

Controllability of time-variant Boolean control networks and its application to Boolean control networks with finite memories

This paper investigates the controllability of time-variant Boolean control networks(BCNs).For the time-variant BCNs,a necessary and sufcient condition for the controllability is given,and a control design algorithm is presented.For a BCN with fnite memories,an equivalent transformation to a time-variant BCN is constructed.Then a necessary and sufcient condition for the controllability and a control design algorithm are obtained.     

Analysis of associative memories based on cellular neural networks with value-varying templates

ABSTRACT

In many research literatures, the dynamical behaviour of cellular neural networks (CNNs) is simplified by using cloning template. However, the flaws of cloning template are obvious, because the correlation between weights of cells in CNNs is enhanced. In order to overcome the shortcomings of cloning template, value-varying templates can be used in CNNs. In this paper, associative memories based on CNNs with value-varying templates are investigated. A criterion about stability of CNNs is presented. Then, the problem about obtaining parameters of CNNs can be translated into a problem of solving linear equations for each cell. A design procedure of associative memories is given by our theories and methods. From the procedure, the parameters of CNNs can be obtained. Finally, three examples are used to demonstrate the effectiveness of our theories and methods. And the results show that success rate of associative memories is higher than previous methods.     

Learning patterns in terms of other patterns

This paper investigates the learning of patterns in terms of previously learned patterns. The main problem is to efficiently select from the set of all previously learned patterns a few in terms of which the pattern to be learned can be described. A computer program for doing this is described and the importance of its memory organization is discussed.     

Transient analysis of packet discarding policies in ATM networks with correlated arrivals

In this paper, we investigate several dynamic congestion control strategies in ATM networks. We present an analytic model to observe the transient behavior of correlated fixed sized cell arrivals into a congested buffer. We derive the transient and time-averaged cell loss probabilities for several packet discarding policies, and we also derive the moments of the first passage time from a congested state to a threshold below which the buffer is considered in a non-congested state. We present numerical results for arrival processes having the same marginal distributions, but differing by autocorrelation coefficient only, thus isolating the effects of correlation on transient behavior. We conjecture how these results affect buffer congestion management procedures.     

Dynamical consequences of fast-rising, slow-decaying synapses in neuronal networks

Synapses that rise quickly but have long persistence are shown to have certain computational advantages. They have some unique mathematical properties as well and in some instances can make neurons behave as if they are weakly coupled oscillators. This property allows us to determine their synchronization properties. Furthermore, slowly decaying synapses allow recurrent networks to maintain excitation in the absence of inputs, whereas faster decaying synapses do not. There is an interaction between the synaptic strength and the persistence that allows recurrent networks to fire at low rates if the synapses are sufficiently slow. Waves and localized structures are constructed in spatially extended networks with slowly decaying synapses.     

Learning in linear neural networks: a survey

Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms.     

Analysis of associative memories based on stability of cellular neural networks with time delay

In the paper, associative memories based on cellular neural networks with time delay are presented. In some previous papers, the relationship between cloning templates is closer and stronger. Therefore, some methods are used to make the relationship loose. First, some theories on stability of cellular neural networks are given. Then, associative memories based on cellular neural networks are given on the basis of these theories. In addition, a design procedure of associative memories is introduced. Finally, some examples are given to verify the theoretical results and design procedures.     

Learning agent influence in MAS with complex social networks

In complex open multi-agent systems (MAS), where there is no centralised control and individuals have equal authority, ensuring cooperative and coordinated behaviour is challenging. Norms and conventions are useful means of supporting cooperation in an emergent decentralised manner, however it takes time for effective norms and conventions to emerge. Identifying influential individuals enables the targeted seeding of desirable norms and conventions, which can reduce the establishment time and increase efficacy. Existing research is limited with respect to considering (i) how to identify influential agents, (ii) the extent to which network location imbues influence on an agent, and (iii) the extent to which different network structures affect influence. In this paper, we propose a methodology for learning a model for predicting the network value of an agent, in terms of the extent to which it can influence the rest of the population. Applying our methodology, we show that exploiting knowledge of the network structure can significantly increase the ability of individuals to influence which convention emerges. We evaluate our methodology in the context of two agent-interaction models, namely, the language coordination domain used by Salazar et al. (AI Communications 23(4): 357–372, 2010) and a coordination game of the form used by Sen and Airiau (in: Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007) with heterogeneous agent learning mechanisms, and on a variety of synthetic and real-world networks. We further show that (i) the models resulting from our methodology are effective in predicting influential network locations, (ii) there are very few locations that can be classified as influential in typical networks, (iii) four single metrics are robustly indicative of influence across a range of network structures, and (iv) our methodology learns which single metric or combined measure is the best predictor of influence in a given network.     

Learning with dynamic and static visualizations: Realistic details only benefit learners with high visuospatial abilities

Learning environments can nowadays easily be enriched with different presentation formats of visualizations, because computer graphics technology is constantly and rapidly developing. This study investigates the effectiveness of dynamic compared to static visualizations. Moreover, the influence of realistic details in the visualizations as well as learners’ prerequisites in terms of their visuospatial abilities were addressed. Eighty university students were randomly assigned to four conditions of a two-by-two between subjects design with the two independent variables dynamism and realism. Learning outcomes were measured by means of a verbal factual knowledge test about the terminology and visuospatial details and a pictorial recognition test about the dynamic processes. Data analyses revealed no effects for factual knowledge. With respect to recognition, learners with dynamic visualizations outperformed learners with static visualizations. Furthermore, there was an interaction between learners’ visuospatial abilities and the degree of realism in the visualization: learners with lower visuospatial abilities showed better recognition with schematized visualizations, whereas learners with higher visuospatial abilities showed better recognition with realistic visualizations. The results imply that when designing instructional materials, both the type of knowledge that has to be acquired as well as learners’ prerequisites such as their visuospatial abilities need to be considered.     

Oscillatory neural networks with self-organized segmentation of overlapping patterns

Temporal coding is considered with an oscillatory network model that generalizes the Cohen-Grossberg-Hopfield model. It is assumed that the frequency of oscillating units increases with stronger and more coherent input. We refer to this mechanism as acceleration. In the context of Hebbian memory, synchronization and acceleration take complementary roles, and their combined effect on the storage of patterns is profound. Acceleration implies the desynchronization that is needed for self-organized segmention of two overlapping patterns. The superposition problem is thereby solved even without including competition couplings. With respect to brain dynamics, we point to analogies with oscillation spindles in the gamma range and responses to perceptual rivalries.