Managing spatio-temporal complexity in Hopfield neural network simulations for large-scale static optimization

A simulation methodology, which trades space complexity with time complexity, to create the Hopfield neural network weight matrix, the costliest data structure for simulation of Hopfield neural network algorithm for large-scale optimization problems, is proposed. Modular composition of a weight term of the Hopfield neural network weight matrix for a generic static optimization problem, which facilitates construction and reconstruction of the weights on demand during a simulation, is exposed. Proposed methodology is demonstrated on a static combinatorial optimization problem, namely the Traveling Salesman Problem (TSP), through the algebraic procedure for temporal (versus spatial) weight matrix construction, pseudo code and C/C++ code implementation, and an associated simulation study. The proposed methodology is successfully tested through simulation on a general purpose Windows™-AMD™ platform for up to 1000 city Traveling Salesman Problem instance, which would require approximately no less than 1TB of memory to be allocated simply to instantiate the weight matrix in the memory space of the simulation process.     

Adaptive and intelligent wireless sensor networks through neural networks: an illustration for infrastructure adaptation through Hopfield network

This paper proposes embedding an artificial neural network into a wireless sensor network in fully parallel and distributed computation mode. The goal is to equip the wireless sensor network with computational intelligence and adaptation capability for enhanced autonomous operation. The applicability and utility of the proposed concept is demonstrated through a case study whereby a Hopfield neural network configured as a static optimizer for the weakly-connected dominating set problem is embedded into a wireless sensor network to enable it to adapt its network infrastructure to potential changes on-the-fly and following deployment in the field. Minimum weakly-connected dominating set defined for the graph model of the wireless sensor network topology is employed to represent the network infrastructure and can be recomputed each time the sensor network topology changes. A simulation study using the TOSSIM emulator for TinyOS-Mica sensor network platform was performed for mote counts of up to 1000. Time complexity, message complexity and solution quality measures were assessed and evaluated for the case study. Simulation results indicated that the wireless sensor network embedded with Hopfield neural network as a static optimizer performed competitively with other local or distributed algorithms for the weakly connected dominating set problem to establish its feasibility.     

Fuzzy Clustering Using A Compensated Fuzzy Hopfield Network

Hopfield neural networks are well known for cluster analysis with an unsupervised learning scheme. This class of networks is a set of heuristic procedures that suffers from several problems such as not guaranteed convergence and output depending on the sequence of input data. In this paper, a Compensated Fuzzy Hopfield Neural Network (CFHNN) is proposed which integrates a Compensated Fuzzy C-Means (CFCM) model into the learning scheme and updating strategies of the Hopfield neural network. The CFCM, modified from Penalized Fuzzy C-Means algorithm (PFCM), is embedded into a Hopfield net to avoid the NP-hard problem and to speed up the convergence rate for the clustering procedure. The proposed network also avoids determining values for the weighting factors in the energy function. In addition, its training scheme enables the network to learn more rapidly and more effectively than FCM and PFCM. In experimental results, the CFHNN method shows promising results in comparison with FCM and PFCM methods.     

Topology constraint free fuzzy gated neural networks for patternrecognition

A novel topology constraint free neural network architecture using a generalized fuzzy gated neuron model is presented for a pattern recognition task. The main feature is that the network does not require weight adaptation at its input and the weights are initialized directly from the training pattern set. The elimination of the need for iterative weight adaptation schemes facilitates quick network set up times which make the fuzzy gated neural networks very attractive. The performance of the proposed network is found to be functionally equivalent to spatio-temporal feature maps under a mild technical condition. The classification performance of the fuzzy gated neural network is demonstrated on a 12-class synthetic three dimensional (3-D) object data set, real-world eight-class texture data set, and real-world 12 class 3-D object data set. The performance results are compared with the classification accuracies obtained from a spatio-temporal feature map, an adaptive subspace self-organizing map, multilayer feedforward neural networks, radial basis function neural networks, and linear discriminant analysis. Despite the network's ability to accurately classify seen data and adequately generalize validation data, its performance is found to be sensitive to noise perturbations due to fine fragmentation of the feature space. This paper also provides partial solutions to the above robustness issue by proposing certain improvements to various modules of the proposed fuzzy gated neural network.     

Design and analysis of an efficient neural network model for solving nonlinear optimization problems

This paper presents an efficient approach based on a recurrent neural network for solving constrained nonlinear optimization. More specifically, a modified Hopfield network is developed, and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points that represent an optimal feasible solution. The main advantage of the developed network is that it handles optimization and constraint terms in different stages with no interference from each other. Moreover, the proposed approach does not require specification for penalty and weighting parameters for its initialization. A study of the modified Hopfield model is also developed to analyse its stability and convergence. Simulation results are provided to demonstrate the performance of the proposed neural network.     

Solving optimization problems with variable-constraint by anextended Cohen-Grossberg model

A variety of real-world problems can be formulated as continuous optimization problems with variable constraint. It is well-known, however, that it is difficult to develop a unified method for obtaining their feasible solutions. We have recognized that the recent work of solving the traveling salesman problem (TSP) by the Hopfield model explores an innovative approach to them as well as combinatorial optimization problems. The Hopfield model is generalized into the Cohen-Grossberg model (CGM) to which a specific Lyapunov function has been found. This paper thus extends the Hopfield method onto the CGM in order to develop a unified solving-method of continuous optimization problems with variable-constraint. Specifically, we consider a certain class of continuous optimization problems with a constraint equation including the Hopfield version of the TSP as a particular member. Then we theoretically develop a method that, from any given problem of that class, derives a network of an extended CGM to provide feasible solutions to it. The main idea for constructing that extended CGM lies in adding to it a synapse dynamical system concurrently operating with its current unit dynamical system so that the constraint equation can be enforced to satisfaction at final states. This construction is also motivated by previous neuron models in biophysics and learning algorithms in neural networks     

Image labelling: a neural network approach

The solution of the image labelling problem using the emerging computational paradigm of neural networks is shown. A brief introduction to neural network technology is provided. The labelling problem is formulated as a problem in symbolic constraint satisfaction. Alternative solution methods are cited. A Hopfield neural network structure which embodies the labelling constraints is developed in detail. The procedure to determine the energy function and interconnection weight is described. Experimental results and network convergence properties are analysed. Future research diections are outlined.     

基于Hopfield网络的多变量动态矩阵控制及其在精馏塔系统中的应用

利用Hopfield网络的优化计算功能求解有约束多变量动态知阵控制问题,算法简单,收敛性好,即或以用硬件实时实现,也可用数值积分求解,对Wood-Berry精馏塔的仿真表明了算法的有效性。     

基于局部进化的Hopfield神经网络的优化计算方法

提出一种基于局部进化的Hopfield神经网络优化计算方法，该方法将遗传算法和Hopfield神经网络结合在一起，克服了Hopfield神经网络易收敛到局部最优值的缺点，以及遗传算法收敛速度慢的缺点。该方法首先由Hopfield神经网络进行状态方程的迭代计算降低网络能量，收敛后的Hopfield神经网络在局部范围内进行遗传算法寻优，以跳出可能的局部最优值陷阱，再由Hopfield神经网络进一步迭代优化。这种局部进化的Hopfield神经网络优化计算方法尤其适合于大规模的优化问题，对图像分割问题和规模较大的200城市旅行商问题的优化计算结果表明，其全局收敛率和收敛速度明显提高。     

Hopfield network with constraint parameter adaptation foroverlapped shape recognition

We propose an energy formulation for homomorphic graph matching by the Hopfield network and a Lyapunov indirect method-based learning approach to adaptively learn the constraint parameter in the energy function. The adaptation scheme eliminates the need to specify the constraint parameter empirically and generates valid and better quality mappings than the analog Hopfield network with a fixed constraint parameter. The proposed Hopfield network with constraint parameter adaptation is applied to match silhouette images of keys and results are presented.     

An efficient approximation algorithm for finding a maximum clique using Hopfield network learning

In this article, we present a solution to the maximum clique problem using a gradient-ascent learning algorithm of the Hopfield neural network. This method provides a near-optimum parallel algorithm for finding a maximum clique. To do this, we use the Hopfield neural network to generate a near-maximum clique and then modify weights in a gradient-ascent direction to allow the network to escape from the state of near-maximum clique to maximum clique or better. The proposed parallel algorithm is tested on two types of random graphs and some benchmark graphs from the Center for Discrete Mathematics and Theoretical Computer Science (DIMACS). The simulation results show that the proposed learning algorithm can find good solutions in reasonable computation time.     

Medical image segmentation using a contextual-constraint-based Hopfield neural cube

Neural-network-based image techniques such as the Hopfield neural networks have been proposed as an alternative approach for image segmentation and have demonstrated benefits over traditional algorithms. However, due to its architecture limitation, image segmentation using traditional Hopfield neural networks results in the same function as thresholding of image histograms. With this technique high-level contextual information cannot be incorporated into the segmentation procedure. As a result, although the traditional Hopfield neural network was capable of segmenting noiseless images, it lacks the capability of noise robustness. In this paper, an innovative Hopfield neural network, called contextual-constraint-based Hopfield neural cube (CCBHNC) is proposed for image segmentation. The CCBHNC uses a three-dimensional architecture with pixel classification implemented on its third dimension. With the three-dimensional architecture, the network is capable of taking into account each pixel's feature and its surrounding contextual information. Besides the network architecture, the CCBHNC also differs from the original Hopfield neural network in that a competitive winner-take-all mechanism is imposed in the evolution of the network. The winner-take-all mechanism adeptly precludes the necessity of determining the values for the weighting factors for the hard constraints in the energy function in maintaining feasible results. The proposed CCBHNC approach for image segmentation has been compared with two existing methods. The simulation results indicate that CCBHNC can produce more continuous, and smoother images in comparison with the other methods.     

一种非线性优化的神经网络

本文提出了用人工神经网络求解具有约束条件的非线性优化问题的具体方法，分析了神经网络能量函数的构成形式，并在常规的Ｈｏｐｆｉｅｌｄ网络模型的基础上构造了一个非全局连接的神经网络动力学模型。这种修改的Ｈｏｐｆｉｅｌｄ网络克服了常规的Ｈｏｐｆｉｅｌｄ网络在求解非线性优化问题时权值不好映射的困难，具有结构清晰，易于软件模拟和硬件实现的优点。     

基于神经网络求解三值模型下的故障诊断问题

在多处理机系统的系统级故障诊断中,一个重要的研究课题是确定最可能故障处理机集,该问题可以归结为NP一完全的整数线性规划问题。连续Hopfietd神经网络能够近似求解最优化问题,因此是解决这类问题的可选路径。文中主要研究如何构建连续Hopfield神经网络,以在三值PMC模型下近似地确定最可能故障集,相比于常用的二值诊断模型,能得到更准确的诊断结果。在超立方体结构上进行了一系列的数值实验,仿真结果表明：该方法具有实用性。     

A neural network approach to job-shop scheduling

A novel analog computational network is presented for solving NP-complete constraint satisfaction problems, i.e. job-shop scheduling. In contrast to most neural approaches to combinatorial optimization based on quadratic energy cost function, the authors propose to use linear cost functions. As a result, the network complexity (number of neurons and the number of resistive interconnections) grows only linearly with problem size, and large-scale implementations become possible. The proposed approach is related to the linear programming network described by D.W. Tank and J.J. Hopfield (1985), which also uses a linear cost function for a simple optimization problem. It is shown how to map a difficult constraint-satisfaction problem onto a simple neural net in which the number of neural processors equals the number of subjobs (operations) and the number of interconnections grows linearly with the total number of operations. Simulations show that the authors' approach produces better solutions than existing neural approaches to job-shop scheduling, i.e. the traveling salesman problem-type Hopfield approach and integer linear programming approach of J.P.S. Foo and Y. Takefuji (1988), in terms of the quality of the solution and the network complexity.     

基于神经网络的工业大系统辨识及稳态递阶优化方法

为了对工业大系统进行稳态递阶优化,必须首先获得系统的稳态模型.从神经网络的分 析人手,给出了工业大系统稳态模型的动态辨识方法及基于神经网络模型的推导方法.为了 提高算法的收敛速度,引入Lagrange函数解决大系统优化问题中的各种约束,并用Hopfield 网络实现了大系统稳态递阶优化的网络算法,最后给出了某一大系统辨识及优化的仿真结果.     

基于Hopfield 神经网络的污水处理过程优化控制

针对前置反硝化污水处理过程的优化控制问题,提出一种基于拉格朗日乘子法的Hofield神经网络优化方法.构造了污水处理过程约束优化问题的数学表达式,通过Hopfield神经网络优化计算生化池第5分区溶解氧浓度和第2分区硝态氮浓度的设定值,并采用PID控制器实现底层的跟踪控制.基于国际标准的Benchmark基准仿真平台进行仿真实验,结果表明污水处理系统在出水关键水质达标的基础上,能够显著降低能耗.     

Discrete particle swarm optimization based on estimation of distribution for terminal assignment problems

Terminal assignment problem (TEAP) is to determine minimum cost links to form a network by connecting a given set of terminals to a given collection of concentrators. This paper presents a novel discrete particle swarm optimization (PSO) based on estimation of distribution (EDA), named DPSO-EDA, for TEAP. EDAs sample new solutions from a probability model which characterizes the distribution of promising solutions in the search space at each generation. The DPSO-EDA incorporates the global statistical information collected from personal best solutions of all particles into the PSO, and therefore each particle has comprehensive learning and search ability. In the DPSO-EDA, a modified constraint handling method based on Hopfield neural network (HNN) is also introduced to fit nicely into the framework of the PSO and thus utilize the merit of the PSO. The DPSO-EDA adopts the asynchronous updating scheme. Further, the DPSO-EDA is applied to a problem directly related to TEAP, the task assignment problem (TAAP), in order to show that the DPSO-EDA can be generalized to other related combinatorial optimization problems. Simulation results on several problem instances show that the DPSO-EDA is better than previous methods.     

FPGA implementation of a systems identification module based upon Hopfield networks

The aim of this contribution is to implement a hardware module that performs parametric identification of dynamical systems. The design is based upon the methodology of optimization with Hopfield neural networks, leading to an adapted version of these networks. An outstanding feature of this modified Hopfield network is the existence of weights that vary with time. Since weights can no longer be stored in read-only memories, these dynamic weights constitute a significant challenge for digital circuits, in addition to the usual issues of area occupation, fixed-point arithmetic and nonlinear functions computations. The implementation, which is accomplished on FPGA circuits, achieves modularity and flexibility, due to the usage of parametric VHDL to describe the network. In contrast to software simulations, the natural parallelism of neural networks is preserved, at a limited cost in terms of circuitry cost and processing time. The functional simulation and the synthesis show the viability of the design. In particular, the FPGA implementation exhibits a reasonably fast convergence, which is required to produce accurate parameter estimations. Current research is oriented towards integrating the estimator within an embedded adaptive controller for autonomous systems.     

基于Hopfield神经网络多用户检测技术的研究

最优多用户检测器的优化问题可以映射为Hopfield神经网络的能量函数的最小化问题。本文对基于典型Hopfield神经网络多用户检测器和基于随机Hopfield神经网络多用户检测器进行了性能分析,计算机仿真表明这两种多用户检测器均具有抗多址干扰和抗远近效应的能力强、运算量小、实时性好等优点。