The hysteretic Hopfield neural network

A new neuron activation function based on a property found in physical systems-hysteresis-is proposed. We incorporate this neuron activation in a fully connected dynamical system to form the hysteretic Hopfield neural network (HHNN). We then present an analog implementation of this architecture and its associated dynamical equation and energy function. We proceed to prove Lyapunov stability for this new model, and then solve a combinatorial optimization problem (i.e., the N-queen problem) using this network. We demonstrate the advantages of hysteresis by showing increased frequency of convergence to a solution, when the parameters associated with the activation function are varied.     

Fuzzy Clustering Using A Compensated Fuzzy Hopfield Network

Hopfield neural networks are well known for cluster analysis with an unsupervised learning scheme. This class of networks is a set of heuristic procedures that suffers from several problems such as not guaranteed convergence and output depending on the sequence of input data. In this paper, a Compensated Fuzzy Hopfield Neural Network (CFHNN) is proposed which integrates a Compensated Fuzzy C-Means (CFCM) model into the learning scheme and updating strategies of the Hopfield neural network. The CFCM, modified from Penalized Fuzzy C-Means algorithm (PFCM), is embedded into a Hopfield net to avoid the NP-hard problem and to speed up the convergence rate for the clustering procedure. The proposed network also avoids determining values for the weighting factors in the energy function. In addition, its training scheme enables the network to learn more rapidly and more effectively than FCM and PFCM. In experimental results, the CFHNN method shows promising results in comparison with FCM and PFCM methods.     

离散时间Hopfield网络的动力系统分析

离散时间的Hopfield网络模型是一个非线性动力系统．对网络的状态变量引入新的能量函数，利用凸函数次梯度性质可以得到网络状态能量单调减少的条件．对于神经元的连接权值且激活函数单调非减(不一定严格单调增加)的Hopfield网络，若神经元激活函数的增益大于权值矩阵的最小特征值，则全并行时渐进收敛；而当网络串行时，只要网络中每个神经元激活函数的增益与该神经元的自反馈连接权值的和大于零即可．同时，若神经元激活函数单调，网络连接权值对称，利用凸函数次梯度的性质，证明了离散时间的Hopfield网络模型全并行时收敛到周期不大于2的极限环．     

Complex-Valued Multistate Associative Memory With Nonlinear Multilevel Functions for Gray-Level Image Reconstruction

A widely used complex-valued activation function for complex-valued multistate Hopfield networks is revealed to be essentially based on a multilevel step function. By replacing the multilevel step function with other multilevel characteristics, we present two alternative complex-valued activation functions. One is based on a multilevel sigmoid function, while the other on a characteristic of a multistate bifurcating neuron. Numerical experiments show that both modifications to the complex-valued activation function bring about improvements in network performance for a multistate associative memory. The advantage of the proposed networks over the complex-valued Hopfield networks with the multilevel step function is more outstanding when a complex-valued neuron represents a larger number of multivalued states. Further, the performance of the proposed networks in reconstructing noisy 256 gray-level images is demonstrated in comparison with other recent associative memories to clarify their advantages and disadvantages.     

Computing association probabilities using parallel Boltzmannmachines

A new computational method is presented for solving the data association problem using parallel Boltzmann machines. It is shown that the association probabilities can be computed with arbitrarily small errors if a sufficient number of parallel Boltzmann machines are available. The probability beta(i)(j) that the i th measurement emanated from the jth target can be obtained simply by observing the relative frequency with which neuron v(i,j) in a two-dimensional network is on throughout the layers. Some simple tracking examples comparing the performance of the Boltzmann algorithm to the exact data association solution and with the performance of an alternative parallel method using the Hopfield neural network are also presented.     

Scheduling multiprocessor job with resource and timing constraintsusing neural networks

The Hopfield neural network is extensively applied to obtaining an optimal/feasible solution in many different applications such as the traveling salesman problem (TSP), a typical discrete combinatorial problem. Although providing rapid convergence to the solution, TSP frequently converges to a local minimum. Stochastic simulated annealing is a highly effective means of obtaining an optimal solution capable of preventing the local minimum. This important feature is embedded into a Hopfield neural network to derive a new technique, i.e., mean field annealing. This work applies the Hopfield neural network and the normalized mean field annealing technique, respectively, to resolve a multiprocessor problem (known to be a NP-hard problem) with no process migration, constrained times (execution time and deadline) and limited resources. Simulation results demonstrate that the derived energy function works effectively for this class of problems.     

Convergence acceleration of the Hopfield neural network byoptimizing integration step sizes

In our previous work we have clarified global convergence of the Hopfield neural network and showed, by computer simulations, improvement of solution quality by gradually decreasing the diagonal elements of the coefficient matrix. In this paper, to accelerate convergence of the Hopfield network, at each time step the integration step size is determined dynamically so that at least one component of a variable vector reaches the surface of the hypercube. The computer simulation for the traveling salesman problem and an LSI module placement problem shows that convergence is stabilized and accelerated compared to integration by a constant step size.     

基于离散Hopfield网络的自相关过程控制方法

对于自相关过程的统计控制,传统统计学方法虚发警报的概率较大,而BP人工神经网络方法权值训练困难,灵敏度不高。提出一种基于联想学习与离散Hopfield网络的自相关过程控制方法。不需任何训练样本,通过正交化编码将过程状态以吸引子的形式存储到Hopfield网络中,并利用网络的联想功能来检测自相关过程的阶跃型突变。算例研究表明,与Elman网络和EWMA方法相比,过程正常时,所提方法的平均链长（ARL）分别提高了27.9%和55.0%;过程异常时,所提方法的ARL分别降低了74.1%与81.8%以上。说明了方法的有效性与优越性。     

基于改进Hopfield神经网络的图像特征点匹配算法

针对图像特征点匹配算法的运行时间呈指数增长的问题,提出了一种新的匹配算法NHop.该算法通过加入新的网络输入输出函数、点对间差异的度量和启发式选择目标点的方式,对传统的Hopfield神经网络进行了改进.新算法不仅解决了传统Hopfield神经网络运行时间长、能量函数易陷入局部极小点的问题,而且也有效地实现了图像特征点的匹配.实验结果表明,与传统的Hopfield神经网络相比,NHop算法的匹配速度更快、准确率更高,对于图像特征点的匹配效果更好.     

用神经网络求解机器人动态调度中的近似指派问题

本文利用HOPFIELD神经网络，对机器人动态调度中的近似指派问题提出了合理的神经网络表示方法，给出了网络的能量函数表示法及神经元状态方程，从而得出了机器人动态调度中近似指派问题的快速求解策略，满足了动态调度的实时性要求．本文从理论上论证了所提算法的收敛性．软件仿真结果表明，本文提出的近似指派问题网络求解方法是有效的，计算结果是满意的．     

一种基于抗体网络的免疫算法

阐述了免疫系统抗体网络的机理和特点,深入分析了抗体网络与常用的免疫算法和Hopfield神经网络异同.通过不断更新输入模式(抗原)和采用最优保存策略,将基于克隆选择的竞争学习算子、自动生成网络结构、剪枝算子和低频变异用于进化操作,提出一种新的基于抗体网络的免疫算法,用于函数优化问题.实验结果表明新算法可行有效.与常用的免疫算法、Hopfield神经网络优化算法比较,新算法具有较好的全局搜索能力和较快收敛速度.     

基于Hopfield神经网络的作业车间生产调度方法

该文提出了基于Hopfield神经网络的作业车间生产调度的新方法.文中给出了作业车 间生产调度问题(JSP)的约束条件及其换位矩阵表示,提出了新的包括所有约束条件的计算能 量函数表达式,得到相应的作业车间调度问题的Hopfield神经网络结构与权值解析表达式,并 提出相应的Hopfield神经网络作业车间调度方法.为了避免Hopfield神经网络容易收敛到局部 极小,从而产生非法调度解的缺点,将模拟退火算法应用于Hopfield神经网络求解,使Hopfield 神经网络收敛到计算能量函数的最小值0,从而保证神经网络输出是一个可行调度方案.该文 改进了已有文献中提出的作业调度问题的Hopfield神经网络方法,与已有算法相比,能够保证 神经网络稳态输出为可行的作业车间调度方案.     

A columnar competitive model for solving combinatorial optimization problems

The major drawbacks of the Hopfield network when it is applied to some combinatorial problems, e.g., the traveling salesman problem (TSP), are invalidity of the obtained solutions, trial-and-error setting value process of the network parameters and low-computation efficiency. This letter presents a columnar competitive model (CCM) which incorporates winner-takes-all (WTA) learning rule for solving the TSP. Theoretical analysis for the convergence of the CCM shows that the competitive computational neural network guarantees the convergence to valid states and avoids the onerous procedures of determining the penalty parameters. In addition, its intrinsic competitive learning mechanism enables a fast and effective evolving of the network. The simulation results illustrate that the competitive model offers more and better valid solutions as compared to the original Hopfield network.     

Multiprocessor Task Assignment with Fuzzy Hopfield Neural Network Clustering Technique

Most scheduling applications have been demonstrated as NP-complete problems. A variety of schemes are introduced in solving those scheduling applications, such as linear programming, neural networks, and fuzzy logic. In this paper, a new approach of first analogising a scheduling problem to a clustering problem and then using a fuzzy Hopfield neural network clustering technique to solve the scheduling problem is proposed. This fuzzy Hopfield neural network algorithm integrates fuzzy c-means clustering strategies into a Hopfield neural network. This investigation utilises this new approach to demonstrate the feasibility of resolving a multiprocessor scheduling problem with no process migration and constrained times (execution time and deadline). Each process is regarded as a data sample, and every processor is taken as a cluster. Simulation results illustrate that imposing the fuzzy Hopfield neural network onto the proposed energy function provides an appropriate approach to solving this class of scheduling problem.       

用神经网络求解机器人动态调度中的近似指派问题

本文利用HOPFIELD神经网络，对机器人动态调度中的近似指派问题提出了合理的神经网络表示方法，给出了网络的能量函数表示法及神经元状态方程，从而得出了机器人动态调度中近似指派问题的快速求解策略，满足了动态调度的实时性要求。本文从理论上论证了所提算法的收敛性。软件仿真结果表明，本文提出的近似指派问题网络求解方法是有效的，计算结果是满意的。     

Cooperative updating in the Hopfield model.

We propose a new method for updating units in the Hopfield model. With this method two or more units change at the same time, so as to become the lowest energy state among all possible states. Since this updating algorithm is based on the detailed balance equation, convergence to the Boltzmann distribution is guaranteed. If our algorithm is applied to finding the minimum energy in constraint satisfaction and combinatorial optimization problems, then there is a faster convergence than those with the usual algorithm in the neural network. This is shown by experiments with the travelling salesman problem, the four-color problem, the N-queen problem, and the graph bi-partitioning problem. In constraint satisfaction problems, for which earlier neural networks are effective in some cases, our updating scheme works fine. Even though we still encounter the problem of ending up in local minima, our updating scheme has a great advantage compared with the usual updating scheme used in combinatorial optimization problems. Also, we discuss parallel computing using our updating algorithm.     

Design and analysis of maximum Hopfield networks

Since McCulloch and Pitts presented a simplified neuron model (1943), several neuron models have been proposed. Among them, the binary maximum neuron model was introduced by Takefuji et al. and successfully applied to some combinatorial optimization problems. Takefuji et al. also presented a proof for the local minimum convergence of the maximum neural network. In this paper we discuss this convergence analysis and show that this model does not guarantee the descent of a large class of energy functions. We also propose a new maximum neuron model, the optimal competitive Hopfield model (OCHOM), that always guarantees and maximizes the decrease of any Lyapunov energy function. Funabiki et al. (1997, 1998) applied the maximum neural network for the n-queens problem and showed that this model presented the best overall performance among the existing neural networks for this problem. Lee et al. (1992) applied the maximum neural network for the bipartite subgraph problem showing that the solution quality was superior to that of the best existing algorithm. However, simulation results in the n-queens problem and in the bipartite subgraph problem show that the OCHOM is much superior to the maximum neural network in terms of the solution quality and the computation time.     

基于Tanh多层函数的量子神经网络算法及其应用的研究

该文提出一种新的改进激励函数的量子神经网络模型。首先为了提高学习速率,在网络权值训练过程中引入了动量项。然后为了有效实现相邻类之间具有覆盖和不确定边界的分类问题,新网络采用区分度更大的双曲正切函数的叠加作为其隐层激励函数。最后将该算法用于字符识别,将双曲正切激励函数的量子神经网络应用于数字、字母和汉字样本的多次实验,并且与原多层激励函数量子神经网络和BP网络的实验效果进行比较,发现改进后量子神经网络不仅具有较高的识别率,而且在样本训练次数上相对原多层激励函数量子神经网络有明显减少。仿真结果证明该方法的优越性。     

New stability conditions for Hopfield networks in partial simultaneous update mode.

Cernuschi-Frias proposed (IEEE Trans. Syst., Man, Cybern., vol.19, p.887-8, 1989) a partial simultaneous updating (PSU) mode for Hopfield networks. He also derived sufficient conditions to ensure global stability. In this letter, a counter-example is given to illustrate that the PSU sequence may converge to limited cycles even if one uses a connection matrix satisfying the Cernuschi-Frias conditions. Then, new sufficient conditions ensuring global convergence of a Hopfield network in PSU mode are derived. Compared with the result of fully parallel mode case, the new result permits a little relaxation on the lower bound of the main diagonal elements of the connection matrix.     

一种基于遗传Hopfield神经网络求解TSP问题的算法

针对Hopfield网络求解TSP问题时出现无效解和收敛性能差的问题,对约束条件能量函数进行改进,构造了一种求解TSP问题的遗传Hopfield神经网络算法,并与经典Hopfield神经网络求解TSP方法进行对比.实验结果表明,本文算法具有更好的整体求解性能.