Convergence of non-autonomous discrete-time Hopfield model with delays

This paper is concerned with boundedness, convergence of solution of a class of non-autonomous discrete-time delayed Hopfield neural network model. Using the inequality technique, we obtain some sufficient conditions ensuring the boundedness of solutions of the discrete-time delayed Hopfield models in time-varying situation. Then, by exploring intrinsic features between non-autonomous system and its asymptotic equations, several novel sufficient conditions are established to ensure that all solutions of the networks converge to the solution of its asymptotic equations. Especially, for case of asymptotic autonomous system or asymptotic periodic system, we obtain some sufficient conditions ensuring all solutions of original system convergent to equilibrium or periodic solution of asymptotic system, respectively. An example is provided for demonstrating the effectiveness of the global stability conditions presented. Our results are not only presented in terms of system parameters and can be easily verified but also are less restrictive than previously known criteria.     

Cooperative updating in the Hopfield model.

We propose a new method for updating units in the Hopfield model. With this method two or more units change at the same time, so as to become the lowest energy state among all possible states. Since this updating algorithm is based on the detailed balance equation, convergence to the Boltzmann distribution is guaranteed. If our algorithm is applied to finding the minimum energy in constraint satisfaction and combinatorial optimization problems, then there is a faster convergence than those with the usual algorithm in the neural network. This is shown by experiments with the travelling salesman problem, the four-color problem, the N-queen problem, and the graph bi-partitioning problem. In constraint satisfaction problems, for which earlier neural networks are effective in some cases, our updating scheme works fine. Even though we still encounter the problem of ending up in local minima, our updating scheme has a great advantage compared with the usual updating scheme used in combinatorial optimization problems. Also, we discuss parallel computing using our updating algorithm.     

Distinguishing features of a small Hopfield model with clipping of synapses

To estimate the memory storage as well as the operational reliability of a small Hopfield neural network model with clipping of synapses, the combinatorial methods have been applied. It has been demonstrated that the recognition error probability depends on parity\oddness of etalons number, which causes reliability to decrease sawtooth-like rather than monotonously. It had been also shown that the reliability of the network based on odd number of the etalons can be increased by adding a random etalon.     

Storage capacity diverges with synaptic efficiency in an associative memory model with synaptic delay and pruning

It is known that storage capacity per synapse increases by synaptic pruning in the case of a correlation-type associative memory model. However, the storage capacity of the entire network then decreases. To overcome this difficulty, we propose decreasing the connectivity while keeping the total number of synapses constant by introducing delayed synapses. In this paper, a discrete synchronous-type model with both delayed synapses and their prunings is discussed as a concrete example of the proposal. First, we explain the Yanai-Kim theory by employing statistical neurodynamics. This theory involves macrodynamical equations for the dynamics of a network with serial delay elements. Next, considering the translational symmetry of the explained equations, we rederive macroscopic steady-state equations of the model by using the discrete Fourier transformation. The storage capacities are analyzed quantitatively. Furthermore, two types of synaptic prunings are treated analytically: random pruning and systematic pruning. As a result, it becomes clear that in both prunings, the storage capacity increases as the length of delay increases and the connectivity of the synapses decreases when the total number of synapses is constant. Moreover, an interesting fact becomes clear: the storage capacity asymptotically approaches 2//spl pi/ due to random pruning. In contrast, the storage capacity diverges in proportion to the logarithm of the length of delay by systematic pruning and the proportion constant is 4//spl pi/. These results theoretically support the significance of pruning following an overgrowth of synapses in the brain and may suggest that the brain prefers to store dynamic attractors such as sequences and limit cycles rather than equilibrium states.     

一种模糊Hopfield网络的学习算法及性质

摘要现基于TL-模Max-TL模糊Hopfield网络(Max-TL FHNN)提出了一种有效的学习算法。对任意给定的模式集合,该学习算法总能找到使该模式集合成为Max-TL FHNN的平衡点集合的所有连接权矩阵中的最大者。任意给定的模式集合都能作为Max-TL FHNN网络的平衡点集合且能使Max-TL FHNN对任意输入在一步内就进入稳定状态,同时该网络对训练模式的摄动具有好的鲁棒性。     

Properties of synaptic transmission and the global stability of delayed activity states

The influence of synaptic channel properties on the stability of delayed activity maintained by recurrent neural networks is studied. The duration of excitatory post-synaptic current (EPSC) is shown to be essential for the global stability of the delayed response. The NMDA receptor channel is a much more reliable mediator of the reverberating activity than the AMPA receptor, due to a longer EPSC. This allows one to interpret the deterioration of the working memory observed in NMDA channel blockade experiments. The key mechanism leading to the decay of the delayed activity originates in the unreliability of synaptic transmission. The optimum fluctuation of the synaptic currents leading to the decay is identified. The decay time is calculated analytically and the result is confirmed computationally.     

Credibility decay model in temporal evidence combination

Data fusion in time domain is sequential and dynamic. Methods to deal with evidence conflict in spatial domain may not suitable in temporal domain. It is significant to determine the dynamic credibility of evidence in time domain. The Markovian requirement of time domain fusion is analyzed based on Dempster's combination rule and evidence discount theory. And the credibility decay model is presented to get the dynamic evidence credibility. Then the evidence is discounted by dynamic discount factor. It's illustrated that such model can satisfied the requirement of data fusion in time domain. Proper and solid decision can be made by this approach.     

Approximating maximum clique with a Hopfield network

In a graph, a clique is a set of vertices such that every pair is connected by an edge. MAX-CLIQUE is the optimization problem of finding the largest clique in a given graph and is NP-hard, even to approximate well. Several real-world and theory problems can be modeled as MAX-CLIQUE. In this paper, we efficiently approximate MAX-CLIQUE in a special case of the Hopfield network whose stable states are maximal cliques. We present several energy-descent optimizing dynamics; both discrete (deterministic and stochastic) and continuous. One of these emulates, as special cases, two well-known greedy algorithms for approximating MAX-CLIQUE. We report on detailed empirical comparisons on random graphs and on harder ones. Mean-field annealing, an efficient approximation to simulated annealing, and a stochastic dynamics are the narrow but clear winners. All dynamics approximate much better than one which emulates a "naive" greedy heuristic.     

A modified Hopfield neural networks model for graphs-based kinematic structure design

Graph theory can be used efficiently for both kinematic and dynamics analysis of mechanical structures. One of the most important and difficult issues in graphs theory-based structures design is graphs isomorphism discernment. The problem is vital for graph theory-based kinematic structures enumeration, which is known to be nondeterministic polynomial-complete problem. To solve the problem, a Hopfield neural networks (HNN) model is presented and some operators are improved to prevent premature convergence. By comparing with genetic algorithm, the computation times of the HNN model shows less affection when the number of nodes were enhanced. It is concluded that the algorithm presented in this paper is efficient for large-scale graphs isomorphism problem.     

Learning node labels with multi-category Hopfield networks

In several real-world node label prediction problems on graphs, in fields ranging from computational biology to World Wide Web analysis, nodes can be partitioned into categories different from the classes to be predicted, on the basis of their characteristics or their common properties. Such partitions may provide further information about node classification that classical machine learning algorithms do not take into account. We introduce a novel family of parametric Hopfield networks (m-category Hopfield networks) and a novel algorithm (Hopfield multi-category—HoMCat), designed to appropriately exploit the presence of property-based partitions of nodes into multiple categories. Moreover, the proposed model adopts a cost-sensitive learning strategy to prevent the remarkable decay in performance usually observed when instance labels are unbalanced, that is, when one class of labels is highly underrepresented than the other one. We validate the proposed model on both synthetic and real-world data, in the context of multi-species function prediction, where the classes to be predicted are the Gene Ontology terms and the categories the different species in the multi-species protein network. We carried out an intensive experimental validation, which on the one hand compares HoMCat with several state-of-the-art graph-based algorithms, and on the other hand reveals that exploiting meaningful prior partitions of input data can substantially improve classification performances.     

动态突触型Hopfield神经网络的动态特性研究

提出一种基于动态突触的离散型Hoppfield神经网（DSDNN）模型,给出了DSDNN的连接权值的动态演化模型及其神经元的状态更新模型．证明了DSDNN的平衡点与常规离散型Hopfield神经网络的平衡点具有一一对应的关系,分析了平衡点的稳定性．最后通过仿真分析了DSDNN的动态演化特性与其参数的关系。     

基于局部进化的Hopfield神经网络的优化计算方法

提出一种基于局部进化的Hopfield神经网络优化计算方法，该方法将遗传算法和Hopfield神经网络结合在一起，克服了Hopfield神经网络易收敛到局部最优值的缺点，以及遗传算法收敛速度慢的缺点。该方法首先由Hopfield神经网络进行状态方程的迭代计算降低网络能量，收敛后的Hopfield神经网络在局部范围内进行遗传算法寻优，以跳出可能的局部最优值陷阱，再由Hopfield神经网络进一步迭代优化。这种局部进化的Hopfield神经网络优化计算方法尤其适合于大规模的优化问题，对图像分割问题和规模较大的200城市旅行商问题的优化计算结果表明，其全局收敛率和收敛速度明显提高。     

A Hopfield neural network with multi-compartmental activation

The Hopfield network is a form of recurrent artificial neural network. To satisfy demands of artificial neural networks and brain activity, the networks are needed to be modified in different ways. Accordingly, it is the first time that, in our paper, a Hopfield neural network with piecewise constant argument of generalized type and constant delay is considered. To insert both types of the arguments, a multi-compartmental activation function is utilized. For the analysis of the problem, we have applied the results for newly developed differential equations with piecewise constant argument of generalized type beside methods for differential equations and functional differential equations. In the paper, we obtained sufficient conditions for the existence of an equilibrium as well as its global exponential stability. The main instruments of investigation are Lyapunov functionals and linear matrix inequality method. Two examples with simulations are given to illustrate our solutions as well as global exponential stability.

     

On zero-order hold equivalents of distributed parameter systems

Various connections are established between linear time-invariant distributed parameter continuous-time systems and their zero-order hold discrete-time equivalents. These connections are established in both the time and frequency domains. The time-domain connections relate various growth constants and norm bounds of the continuous-time systems considered to those of their zero-order hold discrete-time equivalents. The frequency-domain connection provides an upper bound on the difference between the frequency response of a continuous-time system and that of its zero-order hold discrete-time equivalent     

Markovian jump delayed Hopfield networks with multiplicative noise

A non-symmetric version of Hopfield networks subject to state-multiplicative noise, pure time delay and Markov jumps is considered. Such networks arise in the context of visuo-motor control loops and may, therefore, be used to mimic their complex behavior. In this paper, we adopt the Lur’e-Postnikov systems approach to analyze the stochastic stability and the L₂ gain of generalized Hopfield networks including these effects.     

Stabilization of stochastic Hopfield neural network with distributed parameters

In this paper, the stability of stochastic Hopfield neural network with distributed parameters is studied. To discuss the stability of systems, the main idea is to integrate the solution to systems in the space variable. Then, the integration is considered as the solution process of corresponding neural networks described by stochastic ordinary differential equations. A Lyapunov function is constructed and Ito formula is employed to compute the derivative of the mean Lyapunov function along the systems, with respect to the space variable. It is difficult to treat stochastic systems with distributed parameters since there is no corresponding Ito formula for this kind of system. Our method can overcome this difficulty. Till now, the research of stability and stabilization of stochastic neural networks with distributed parameters has not been considered.     

时滞脉冲型Hopfield神经网络的全局指数稳定性*

基于众多领域及生物神经网络本身所存在的脉冲瞬动现象，本文首次提出并研究了带时滞的脉冲型Ｈｏｐｆｉｅｌｄ神经网络的全局指定稳定性问题，并讨论了其平衡态的存在唯一性。     

Mathematical improvement of the Hopfield model for feasiblesolutions to the traveling salesman problem by a synapse dynamicalsystem

It is well known that the Hopfield Model (HM) for neural networks to solve the Traveling Salesman Problem (TSP) suffers from three major drawbacks. (1) It can converge on nonoptimal locally minimum solutions. (2) It can converge on infeasible solutions. (3) Results are very sensitive to the careful tuning of its parameters. A number of methods have been proposed to overcome (a) well. In contrast, work on (b) and (c) has not been sufficient; techniques have not been generalized to more general optimization problems. Thus this paper mathematically resolves (b) and (c) to such an extent that the resolution can be applied to solving with some general network continuous optimization problems including the Hopfield version of the TSP. It first constructs an Extended HM (E-HM) that overcomes both (b) and (c). Fundamental techniques of the E-HM lie in the addition of a synapse dynamical system cooperated with the current HM unit dynamical system. It is this synapse dynamical system that makes the TSP constraint hold at any final states for whatever choices of the IIM parameters and an initial state. The paper then generalizes the E-HM further to a network that can solve a class of continuous optimization problems with a constraint equation where both of the objective function and the constraint function are nonnegative and continuously differentiable.