A neural network approach to job-shop scheduling

A novel analog computational network is presented for solving NP-complete constraint satisfaction problems, i.e. job-shop scheduling. In contrast to most neural approaches to combinatorial optimization based on quadratic energy cost function, the authors propose to use linear cost functions. As a result, the network complexity (number of neurons and the number of resistive interconnections) grows only linearly with problem size, and large-scale implementations become possible. The proposed approach is related to the linear programming network described by D.W. Tank and J.J. Hopfield (1985), which also uses a linear cost function for a simple optimization problem. It is shown how to map a difficult constraint-satisfaction problem onto a simple neural net in which the number of neural processors equals the number of subjobs (operations) and the number of interconnections grows linearly with the total number of operations. Simulations show that the authors' approach produces better solutions than existing neural approaches to job-shop scheduling, i.e. the traveling salesman problem-type Hopfield approach and integer linear programming approach of J.P.S. Foo and Y. Takefuji (1988), in terms of the quality of the solution and the network complexity.     

Cooperative updating in the Hopfield model.

We propose a new method for updating units in the Hopfield model. With this method two or more units change at the same time, so as to become the lowest energy state among all possible states. Since this updating algorithm is based on the detailed balance equation, convergence to the Boltzmann distribution is guaranteed. If our algorithm is applied to finding the minimum energy in constraint satisfaction and combinatorial optimization problems, then there is a faster convergence than those with the usual algorithm in the neural network. This is shown by experiments with the travelling salesman problem, the four-color problem, the N-queen problem, and the graph bi-partitioning problem. In constraint satisfaction problems, for which earlier neural networks are effective in some cases, our updating scheme works fine. Even though we still encounter the problem of ending up in local minima, our updating scheme has a great advantage compared with the usual updating scheme used in combinatorial optimization problems. Also, we discuss parallel computing using our updating algorithm.     

An asynchronous recurrent linear threshold network approach to solving the traveling salesman problem

In this paper, an approach to solving the classical Traveling Salesman Problem (TSP) using a recurrent network of linear threshold (LT) neurons is proposed. It maps the classical TSP onto a single-layered recurrent neural network by embedding the constraints of the problem directly into the dynamics of the network. The proposed method differs from the classical Hopfield network in the update of state dynamics as well as the use of network activation function. Furthermore, parameter settings for the proposed network are obtained using a genetic algorithm, which ensure a stable convergence of the network for different problems. Simulation results illustrate that the proposed network performs better than the classical Hopfield network for optimization.     

Multiprocessor Task Assignment with Fuzzy Hopfield Neural Network Clustering Technique

Most scheduling applications have been demonstrated as NP-complete problems. A variety of schemes are introduced in solving those scheduling applications, such as linear programming, neural networks, and fuzzy logic. In this paper, a new approach of first analogising a scheduling problem to a clustering problem and then using a fuzzy Hopfield neural network clustering technique to solve the scheduling problem is proposed. This fuzzy Hopfield neural network algorithm integrates fuzzy c-means clustering strategies into a Hopfield neural network. This investigation utilises this new approach to demonstrate the feasibility of resolving a multiprocessor scheduling problem with no process migration and constrained times (execution time and deadline). Each process is regarded as a data sample, and every processor is taken as a cluster. Simulation results illustrate that imposing the fuzzy Hopfield neural network onto the proposed energy function provides an appropriate approach to solving this class of scheduling problem.       

随机神经网络发展现状综述

随机神经网络 (RNN)在人工神经网络中是一类比较独特、出现较晚的神经网络 ,它的网络结构、学习算法、状态更新规则以及应用等方面都因此具有自身的特点 .作为仿生神经元数学模型 ,随机神经网络在联想记忆、图像处理、组合优化问题上都显示出较强的优势 .在阐述随机神经网络发展现状、网络特性以及广泛应用的同时 ,专门将RNN分别与Hopfield网络、模拟退火算法和Boltzmann机在组合优化问题上的应用进行了分析对比 ,指出RNN是解决旅行商 (TSP)等问题的有效途径     

基于多目标蚁群优化的交通路径优化研究

路径优化是智能交通网络的重要组成部分。如今,仅仅要求出发地与目的地之间的距离最短在实际交通网络中已经不能满足人们的出行需求。本文引入危险品运输(transportation of dangerous goods)概念建立多目标路径优化模型。同时采用蚁群优化算法(Ant colony algorithm,ACA)作为解决多目标优化问题的方法。在分析蚂蚁算法运行机理的基础上,应用MAXMIN方法解决多目标优化模型中候选解的评价问题,并以MAXMIN方法得出的解的适应度(fitness)作为参数改进信息素定义规则,指导蚂蚁算法的搜索方向。最后,在GIS(Geographical Information System)决策系统的支持下,把该模型和算法应用于香港路径优化的实际问题中。实验结果表明模型是有效的,优化算法的收敛速度和优化结果都达到了预期效果。     

一种基于抗体网络的免疫算法

阐述了免疫系统抗体网络的机理和特点,深入分析了抗体网络与常用的免疫算法和Hopfield神经网络异同.通过不断更新输入模式(抗原)和采用最优保存策略,将基于克隆选择的竞争学习算子、自动生成网络结构、剪枝算子和低频变异用于进化操作,提出一种新的基于抗体网络的免疫算法,用于函数优化问题.实验结果表明新算法可行有效.与常用的免疫算法、Hopfield神经网络优化算法比较,新算法具有较好的全局搜索能力和较快收敛速度.     

Design and analysis of an efficient neural network model for solving nonlinear optimization problems

This paper presents an efficient approach based on a recurrent neural network for solving constrained nonlinear optimization. More specifically, a modified Hopfield network is developed, and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points that represent an optimal feasible solution. The main advantage of the developed network is that it handles optimization and constraint terms in different stages with no interference from each other. Moreover, the proposed approach does not require specification for penalty and weighting parameters for its initialization. A study of the modified Hopfield model is also developed to analyse its stability and convergence. Simulation results are provided to demonstrate the performance of the proposed neural network.     

传感器网络中单源单汇路由问题的模型与算法

路由问题是无线传感器网络中的核心问题之一,寻找从源到汇的最小费用路径非常困难。蚁群优化算法是最近提出的求解复杂组合优化问题的启发式算法,该算法能够在完全分布式环境下对复杂问题进行求解。文章建立了无线传感器网络中单源单汇路由问题的数学模型,并给出了基于蚁群优化的求解算法。     

Genetic programming over context-free languages with linear constraints for the knapsack problem: first results

In this paper, we introduce genetic programming over context-free languages with linear constraints for combinatorial optimization, apply this method to several variants of the multidimensional knapsack problem, and discuss its performance relative to Michalewicz's genetic algorithm with penalty functions. With respect to Michalewicz's approach, we demonstrate that genetic programming over context-free languages with linear constraints improves convergence. A final result is that genetic programming over context-free languages with linear constraints is ideally suited to modeling complementarities between items in a knapsack problem: The more complementarities in the problem, the stronger the performance in comparison to its competitors.     

A portable and scalable algorithm for a class of constrained combinatorial optimization problems

This paper presents a portable and scalable approach for a class of constrained combinatorial optimization problems (CCOPs) which requires to satisfy a set of constraints and to optimize and objective function simultaneously. In particular, this paper is focused on the class of CCOPs that admits a representation in terms of a square matrix of constraints C.The algorithm consists of a hybrid neural-genetic algorithm, formed by a Hopfield Neural Network (HNN) which solves the problem's constraints, and a Genetic Algorithm (GA) for optimizing the objective function. This separated management of constraints and optimization procedures makes the proposed algorithm scalable and robust. The portability of the algorithm is given by the fact that the HNN dynamics depends only on the matrix C of constraints.We show these properties of scalability and portability by solving three different CCOPs with our algorithm, the frequency assignment problem in a mobile telecommunications network, the reduction of the interference in satellite systems and the design of FPGAs with segmented channel routing architecture. We compare our results with previous approaches to these problems, obtaining very good results in all of them.     

Two-stage updating pheromone for invariant ant colony optimization algorithm

Ant colony optimization (ACO) is a metaheuristic approach for combinatorial optimization problems. With the introduction of hypercube framework, invariance property of ACO algorithms draws more attention. In this paper, we propose a novel two-stage updating pheromone for invariant ant colony optimization (TSIACO) algorithm. Compared with standard ACO algorithms, TSIACO algorithm uses solution order other than solution itself as independent variable for quality function. In addition, the pheromone trail is updated with two stages: in one stage, the first r iterative optimal solutions are employed to enhance search capability, and in another stage, only optimal solution is used to accelerate the speed of convergence. And besides, the pheromone value is limited to an interval. We prove that TSIACO not only has the property of linear transformational invariance but also has translational invariance. We also prove that the pheromone trail can limit to the interval (0, 1]. Computational results on the traveling salesman problem show the effectiveness of TSIACO algorithm.     

Combining tabu Hopfield network and estimation of distribution for unconstrained binary quadratic programming problem

Unconstrained binary quadratic programming problem (UBQP) consists in maximizing a quadratic 0–1 function. It is a well known NP-hard problem and is considered as a unified model for a variety of combinatorial optimization problems. This paper combines a tabu Hopfield neural network (HNN) (THNN) with estimation of distribution algorithm (EDA), and thus a THNN–EDA is proposed for the UBQP. In the THNN, the tabu rule, instead of the original updating rule of the HNN, is used to govern the state transition or updating of neurons to search for the global minimum of the energy function. A probability vector in EDA model is built to characterize the distribution of promising solutions in the search space, and then the THNN is guided by the global search information in EDA model to search better solution in the promising region. Thus, the short term memory of the tabu mechanism in the THNN cooperates with the long term memory mechanism in the EDA to help the network escape from local minima. The THNN–EDA is tested on 21 UBQP benchmark problems with the size ranging from 3000 to 7000, and 48 maximum cut benchmark problems, a special case of the UBQP, with the size ranging from 512 to 3375. Simulation results show that the THNN–EDA is better than the other HNN based algorithms, and is better than or competitive with metaheuristic algorithms and state-of-the-art algorithms.     

Interval-indexed formulation based heuristics for single machine total weighted tardiness problem

In this paper, we solve the single machine total weighted tardiness problem by using integer programming and linear programming based heuristic algorithms. Interval-indexed formulation is used to formulate the problem. We discuss several methods to form the intervals and different post-processing methods. Then, we show how our algorithm can be used to improve a population of a genetic algorithm. We also provide some computational results that show the effectiveness of our algorithm. Many aspects of our heuristic algorithm are quite general and can be applied to other scheduling and combinatorial optimization problems.     

Optimal matching by the transiently chaotic neural network

Dynamic programming matching (DPM) is a technique that finds an optimal match between two sequences of feature vectors allowing for stretched and compressed sections of the sequence. The purpose of this study is to formulate the matching problem as an optimization task and carry out this optimization problem by means of a chaotic neural network. The proposed method uses TCNN, a Hopfield neural network with decaying self-feedback, to find the best-matching (i.e., the lowest global distance) path between an input and a template. Experimental results show a very good performance for the proposed algorithm in pattern recognition tasks.     

OPTIMIZATION OF SPARE CAPACITY IN WDM NETWORKS FOR REAL-TIME SERVICE RESTORATION

In the case of network malfunction a network with restoration capability requires spare capacity to be used. Optimization of the spare capacity in this case is to find the minimum amount of spare capacity for the network to survive from network component failures. In this paper, the optimization of the spare capacity problem is investigated for the wavelength division multiplexing (WDM) mesh networks without wavelength conversion. To minimize the spare capacity, we will optimize both the routing and the wavelength assignment. This combinatorial problem is usually called the routing and wavelength assignment (RWA) problem and it is well known to be NP-hard. We give an integer linear programming (ILP) formulation for the problem. Due to the excessive run-times of the ILP, we propose a hybrid genetic algorithm approach (GA) for the problem. For benchmarking purpose, simulated annealing (SA) and Tabu search (TS) are also applied to this problem. To validate the effectiveness of the proposed method, the approach is applied to the China network, which has a more complicated network topology. Simulation results are very favorable to the GA approach.     

An algorithm based on ant colony optimization for the minimum connected dominating set problem

Ant colony optimization is a well established metaheuristic from the swarm intelligence field for solving difficult optimization problems. In this work we present an application of ant colony optimization to the minimum connected dominating set problem, which is an NP-hard combinatorial optimization problem. Given an input graph, valid solutions are connected subgraphs of the given input graph. Due to the involved connectivity constraints, out-of-the-box integer linear programming solvers do not perform well for this problem. The developed ant colony optimization algorithm uses reduced variable neighborhood search as a sub-routine. Moreover, it can be applied to the weighted and to the non-weighted problem variants. An extensive experimental evaluation presents the comparison of our algorithm with the respective state-of-the-art techniques from the literature. It is shown that the proposed algorithm outperforms the current state of the art for both problem variants. For comparison purposes we also develop a constraint programming approach based on graph variables. Even though its performance deteriorates with growing instance size, it performs surprisingly well, solving 315 out of 481 considered problem instances to optimality.     

“Optimal” Hopfield network for combinatorialoptimization with linear cost function

An "optimal" Hopfield network is presented for combinatorial optimization problems with linear cost function. It is proved that a vertex of the network state hypercube is asymptotically stable if and only if it is an optimal solution to the problem. That is, one can always obtain an optimal solution whenever the network converges to a vertex. In this sense, this network can be called the "optimal" Hopfield network. It is also shown through simulations of assignment problems that this network obtains optimal or nearly optimal solutions more frequently than other familiar Hopfield networks.     

Solving optimization problems with variable-constraint by anextended Cohen-Grossberg model

A variety of real-world problems can be formulated as continuous optimization problems with variable constraint. It is well-known, however, that it is difficult to develop a unified method for obtaining their feasible solutions. We have recognized that the recent work of solving the traveling salesman problem (TSP) by the Hopfield model explores an innovative approach to them as well as combinatorial optimization problems. The Hopfield model is generalized into the Cohen-Grossberg model (CGM) to which a specific Lyapunov function has been found. This paper thus extends the Hopfield method onto the CGM in order to develop a unified solving-method of continuous optimization problems with variable-constraint. Specifically, we consider a certain class of continuous optimization problems with a constraint equation including the Hopfield version of the TSP as a particular member. Then we theoretically develop a method that, from any given problem of that class, derives a network of an extended CGM to provide feasible solutions to it. The main idea for constructing that extended CGM lies in adding to it a synapse dynamical system concurrently operating with its current unit dynamical system so that the constraint equation can be enforced to satisfaction at final states. This construction is also motivated by previous neuron models in biophysics and learning algorithms in neural networks     

Transient chaotic neural network-based disjoint multipath routing for mobile ad-hoc networks

Due to mobility of wireless hosts, routing in mobile ad-hoc networks (MANETs) is a challenging task. Multipath routing is employed to provide reliable communication, load balancing, and improving quality of service of MANETs. Multiple paths are selected to be node-disjoint or link-disjoint to improve transmission reliability. However, selecting an optimal disjoint multipath set is an NP-complete problem. Neural networks are powerful tools for a wide variety of combinatorial optimization problems. In this study, a transient chaotic neural network (TCNN) is presented as multipath routing algorithm in MANETs. Each node in the network can be equipped with a neural network, and all the network nodes can be trained and used to obtain optimal or sub-optimal high reliable disjoint paths. This algorithm can find both node-disjoint and link-disjoint paths with no extra overhead. The simulation results show that the proposed method can find the high reliable disjoint path set in MANETs. In this paper, the performance of the proposed algorithm is compared to the shortest path algorithm, disjoint path set selection protocol algorithm, and Hopfield neural network (HNN)-based model. Experimental results show that the disjoint path set reliability of the proposed algorithm is up to 4.5 times more than the shortest path reliability. Also, the proposed algorithm has better performance in both reliability and the number of paths and shows up to 56% improvement in path set reliability and up to 20% improvement in the number of paths in the path set. The proposed TCNN-based algorithm also selects more reliable paths as compared to HNN-based algorithm in less number of iterations.