Optimization via intermittency with a self-organizing neural network

One of the major obstacles in using neural networks to solve combinatorial optimization problems is the convergence toward one of the many local minima instead of the global minima. In this letter, we propose a technique that enables a self-organizing neural network to escape from local minima by virtue of the intermittency phenomenon. It gives rise to novel search dynamics that allow the system to visit multiple global minima as meta-stable states. Numerical experiments performed suggest that the phenomenon is a combined effect of Kohonen-type competitive learning and the iterated softmax function operating near bifurcation. The resultant intermittent search exhibits fractal characteristics when the optimization performance is at its peak in the form of 1/f signals in the time evolution of the cost, as well as power law distributions in the meta-stable solution states. TheN-Queens problem is used as an example to illustrate the meta-stable convergence process that sequentially generates, in a single run, 92 solutions to the 8-Queens problem and 4024 solutions to the 17-Queens problem.     

RanGen: A Random Network Generator for Activity-on-the-Node Networks

In this paper, we describe RanGen, a random network generator for generating activity-on-the-node networks and accompanying data for different classes of project scheduling problems. The objective is to construct random networks which satisfy preset values of the parameters used to control the hardness of a problem instance. Both parameters which are related to the network topology and resource-related parameters are implemented. The network generator meets the shortcomings of former network generators since it employs a wide range of different parameters which have been shown to serve as possible predictors of the hardness of different project scheduling problems. Some of them have been implemented in former network generators while others have not.     

An efficient neurodynamic model to solve nonconvex nonlinear optimization problems and its applications

This paper presents a recurrent neural network for solving nonconvex nonlinear optimization problems subject to nonlinear inequality constraints. First, the p-power transformation is exploited for local convexification of the Lagrangian function in nonconvex nonlinear optimization problem. Next, the proposed neural network is constructed based on the Karush–Kuhn–Tucker (KKT) optimality conditions and the projection function. An important property of this neural network is that its equilibrium point corresponds to the optimal solution of the original problem. By utilizing an appropriate Lyapunov function, it is shown that the proposed neural network is stable in the sense of Lyapunov and convergent to the global optimal solution of the original problem. Also, the sensitivity of the convergence is analysed by changing the scaling factors. Compared with other existing neural networks for such problem, the proposed neural network has more advantages such as high accuracy of the obtained solutions, fast convergence, and low complexity. Finally, simulation results are provided to show the benefits of the proposed model, which compare to or outperform existing models.     

H2 norm accumulation and its impact on synchronisation of complex dynamical networks

This article addresses the H ₂ norm accumulation problem of linearly coupled dynamical networks. An interesting outer-coupling relationship is constructed, under which the H ₂ norm of the newly constructed network with column-input and row-output matrices increases exponentially fast in terms of the node number N: it increases generally much faster than 2 ^N when N is large while the H ₂ norm of each node is 1. However, the H ₂ norm of the network with a diffusive coupling is equal to γ₂ N, i.e. increasing only linearly, when the network is stable, where γ₂ is the H ₂ norm of a single node. And the H ₂ norm of the network with antisymmetrical coupling also increases, but rather slowly, with respect to the node number N. Other networks with block-diagonal-input and block-diagonal-output matrices behave similarly. It demonstrates that the changes of H ₂ norms in different networks are very complicated, despite the fact that the network structures are linear. Finally, the influence of the H ₂ norm of the locally linearised network on the output of a network with Lur'e node dynamics is discussed, demonstrating a significant effect of the H ₂ norm on the network output synchronisation behaviours.     

Exact Solutions to Diameter and Routing Problems in PEC Networks

Recently, the diameter problem for packed exponential connections (PEC) networks was addressed by Cho-Chin Lin and V. K. Prasanna [Proc. Symposium on Parallel and Distributed Processing, 1992, pp. 368–375], who presented asymptotically tight bounds for the diameter and showed asymptotically optimal routing algorithms. In this paper exact, solutions to the diameter and routing problems of PEC networks are derived, thereby strengthening the asymptotic bounds of Lin and Prasanna. For anN= 2ⁿnode PEC network, with[formula]an integer, it is shown that the diameter is given by the simple expression[formula]An exact expression for the diameter of PEC networks for generalNis also derived. Efficient algorithms for shortest-path routing between nodes in a PEC network are then developed. These algorithms use at mostO(log²N) time for computing the lengths of minimal routes between nodes. Finally, a simple modification to obtain symmetric PEC networks is suggested.     

基于多目标优化的网络社区发现方法

社区发现是复杂网络挖掘中的重要任务之一,在恐怖组织识别、蛋白质功能预测、舆情分析等方面具有重要的理论和应用价值.但是,现有的社区质量评判指标具有数据依赖性与耦合关联性,而且基于单一评判指标优化的网络社区发现算法有很大的局限性.针对这些问题,将网络社区发现问题形式化为多目标优化问题,提出了一种基于多目标粒子群优化的网络社区发现算法MOCD-PSO,它选取模块度Q、最小最大割MinMaxCut 与轮廓(silhouette)这3 个指标进行综合寻优.实验结果表明,MOCD-PSO 算法具有较好的收敛性,能够发现分布均匀且分散度较高的Pareto 最优网络社区结构集,并且无论与单目标优化方法(GN 与GA-Net)相比较,还是与多目标优化算法(MOGANet与SCAH-MOHSA)相比较,MOCD-PSO 算法都能在无先验信息的条件下挖掘出更高质量的网络社区.     

On necessary conditions for scale-freedom in complex networks,with applications to computer communication systems

Many complex networks exhibit a scale-free, power-law distribution of vertex degrees. This common feature is a consequence of two generic mechanisms relating to the formation of real networks: (i) networks tend to expand over time through the addition of new vertices and (ii) new vertices attach preferentially to those that are already well connected. We show that for many natural or man-made complex networks possessing a scale-free power-law distribution with the exponent γ ≥ 2, the number of degree-1 vertices, when nonzero, is of the same order as the network size N and that the average degree is of order at most log N. Our results expose another necessary characteristic of such networks. Furthermore, our method has the benefit of relying only on conditions that are static and easily verified for arbitrary networks. We use the preceding results to derive a closed-form formula approximating the distance distribution in scale-free networks. Such distributions are applied extensively in the fields of computer communication and software architecture, among other domains.     

Using coroutines in pascal

Coroutines are routines which communicate with each other in a more general way than that provided by the normal subroutine mechanism. Although they have many potential applications, their use has been restricted by their lack of availability in common high-level languages. This paper discusses some of the issues involved in implementing coroutines, and proposes an implementation written in Pascal which may be incorporated into a Pascal program to give coroutine facilities. Use of the system is illustrated by two solutions to the N-Queens problem by different coroutine strategies. The basic system is then extended to allow more advanced use of coroutines.     

A Sigma-Pi-Sigma Neural Network (SPSNN)

This letter presents a sigma-pi-sigma neural network (SPSNN) structure. The SPSNN can learn to implement static mapping that multilayer neural networks and radial basis function networks usually do. The output of the SPSNN has the sum of product-of-sum form , where x _j's are inputs, N _v is the number of inputs, f _nij() is a function to be generated through the network training, and K is the number of pi-sigma network (PSN) which is the basic building block for SPSNN. A linear memory array can be used to implement f _nij (). The function f _nij (x _j ) can be expressed as , where B _ijk() is a single-variable basis function, w _nijk's are weight values stored in memory, N _q is the quantized element number for x _j , and N _e is the number of basis functions in the neighborhood used for storing information for x _j. If all B _ijk()'s are Gaussian functions, the new neural network degenerates to a Gaussian function network. This paper focuses on the use of overlapped rectangular pulses as the basis functions. With such basis functions, will equal either zero or w _nijk, and the computation of f _nij (x _j) becomes a simple addition of retrieved w _nijk's. The new neural network structure demonstrates excellent learning convergence characteristics and requires small memory space. It has merits over multilayer neural networks, radial basis function networks and CMAC. This revised version was published online in June 2006 with corrections to the Cover Date.     

一类实际网络中的最小截算法

讨论了节点和边都有容量限制的无向平面网络中的两点间的最小截问题.传统方法是把节点和边都有容量的网络中的最小截问题转化为只有边有容量的问题,但该方法用在平面网络时不能保持网络的平面性,因此网络的平面性不能得到利用.使用传统方法的计算时间为O(n²logn)(其中n为网络的节点数).给出了可以充分利用网络平面性的方法.对源和汇共面的s-t平面网络,把最小截问题转化为平面图上两点间的最短路径问题,从而可以得到O(n)时间的算法;对一般的平面网络,给出了新的将节点和边都有容量的问题转化为仅边有容量问题的方法,这种转化方法不破坏网络的平面性,从而可以利用平面网络中仅边有容量问题的计算方法,使原问题在O(nlogn)时间内获得解决.     

Properties of variations in the periods of ring neural oscillators with noise

Effects of additive noise on a series of the periods of oscillations in unidirectionally coupled ring neural networks of ring oscillator type are studied. Kinematical models of the traveling waves of an inconsistency, i.e. the successive same signs in the states of adjacent neurons in the network, are derived. A series of the half periods in the network of N neuron is then expressed by the sum of N sequences of the N-first order autoregressive process, the process with the spectrum of exponential type and the first-order autoregressive process. Noise and the interaction of the inconsistency cause characteristic positive correlations in a series of the half periods of the oscillations. Further, an experiment on an analog circuit of the ring neural oscillator was done and it is shown that correlations in the obtained periods of the oscillations agree with the derived three expressions.     

Hopfield Neural Network Based Stereo Matching Algorithm

In this paper, a neural network based optimization method is described in order to solve the problem of stereo matching for a set of primitives extracted from a stereoscopic pair of images. The neural network used is the 2D Hopfield network. The matching problem amounts to the minimization of an energy function involving specified stereoscopic constraints. This function reaches its minimum when these constraints are satisfied. The network converges to its stable state when the minimum is reached. In the initial step, the primitives to match are extracted from the stereoscopic pair of images. The primitives we use are specific points of interest. The feature extraction technique is the one developed by Moravec, and called the interest operator. Its output comprises mostly corners or feature points with high variance. The Hopfield network is represented as a N _l × N _r matrix of neurons, where N _l is the number of features in the left image and N _r the number of features in the right one. An update of the state of each neuron is done in order to perform the network evolution and then allowing it to settle down into a stable state. In the stable state, each neuron represents a possible match between a left candidate and a right one.     

A new framework for consensus for discrete-time directed networks of multi-agents with distributed delays

In this article, the distributed consensus problem is considered for discrete-time delayed networks of dynamic agents with fixed topologies, where the networks under investigation are directed and the time delays involved are distributed time delays including a single or multiple time delay(s) as special cases. By using the invariance principle of delay difference systems, a new unified framework is established to deal with the consensus for the discrete-time delayed multi-agent system. It is shown that the addressed discrete-time network with arbitrary distributed time delays reaches consensus provided that it is strongly connected. A numerical example is presented to illustrate the proposed methods.     

Parallel stereocorrelation on a reconfigurable multi-ring network

A reconfigurable network termed as the reconfigurable multi-ring network (RMRN) is described. The RMRN is shown to be a truly scalable network in that each node in the network has a fixed degree of connectivity and the reconfiguration mechanism ensures a network diameter of O(log₂ N) for anN-processor network. Algorithms for the two-dimensional mesh and the SIMD or SPMD n-cube are shown to map very elegantly onto the RMRN. Basic message passing and reconfiguration primitives for the SIMD/SPMD RMRN are designed for use as building blocks for more complex parallel algorithms. The RMRN is shown to be a viable architecture for image processing and computer vision problems using the parallel computation of the stereocorrelation imaging operation as an example. Stereocorrelation is one of the most computationally intensive imaging tasks. It is used as a visualization tool in many applications, including remote sensing, geographic information systems and robot vision.An earlier version of this paper was presented at the 1995 International Conference on Parallel and Distributed Processing Techniques and Applications.     

Evolutionary Design of Arbitrarily Large Sorting Networks Using Development

An evolutionary algorithm is combined with an application-specific developmental scheme in order to evolve efficient arbitrarily large sorting networks. First, a small sorting network (that we call the embryo) has to be prepared to solve the trivial instance of a problem. Then the evolved program (the constructor) is applied on the embryo to create a larger sorting network (solving a larger instance of the problem). Then the same constructor is used to create a new instance of the sorting network from the created larger sorting network and so on. The proposed approach allowed us to rediscover the conventional principle of insertion which is traditionally used for constructing large sorting networks. Furthermore, the principle was improved by means of the evolutionary technique. The evolved sorting networks exhibit a lower implementation cost and delay.     

Expansion of a matrix in terms of external products of configuration vectors

The paper proposes a neural-net iterative algorithm that allows us to represent any random symmetrical N×N matrix as a weighted Hebbian series of configuration vectors with a given accuracy. The iterative algorithm is shown to demonstrate the fastest convergence when the vectors of expansion are stable nods of the N-dimensional space corresponding to the extremums of the neural-net energy functional. It so proves that all conclusions about neural networks and optimization algorithms that are based on Hebbian matrices are true for any other type of matrix. The text was submitted by the author in English.     

相接行凸约束网络的快速识别算法

约束网络为计算机科学中的许多问题提供了一种有效的表示方法.一般而言,约束满足问题是NP完全的.然而,许多实际问题通常对约束的结构或形式施加了特殊的限制,从而能够高效地加以解决.迄今,为了识别易处理约束类,人们对特殊的约束或约束网络方面进行了许多研究.相接行凸(connected row-convex,简称CRC)约束网络是Deville等人提出的一类易处理问题.为了给该类问题寻求有效的快速识别算法,在CRC约束网络相关工作基础上,提出了CRC约束矩阵的标准型.在分析CRC约束矩阵的标准型性质的基础上,利用行凸(row-convex,简称RC)约束网络的判定,结合PQ树(由P节点和Q节点构成的树)的性质和矩阵的索引表示法,给出了CRC约束网络的快速识别算法.该算法的时间复杂度为O(n³d²),其中,n为约束网络涉及的变量数,d为各变量的定义域中最大定义域的大小.该时间复杂度达到该类问题的最佳时间复杂度,从而为实际的CRC约束满足问题的求解提供了可行的方法.     

A three-stage construction for multiconnection networks

Summary A multiconnection network of size N is a switching network with N inputs and N outputs which realizes multiconnections, i.e., connections between the N inputs and N outputs in such a way that every output is connected to exactly one input, but an input can be connected to an arbitrary number of outputs. That network is complete if it can realize all N ^N multiconnections. This structure generalizes the permutation network. We consider here the design of multiconnection networks by a three-stage Clos network using complete substitution networks as its building cells and we show that the resulting multiconnection network is complete if and only if the cells in the middle stage have size 2. Moreover, we describe the control algorithm for such a network. This leads to the design of cellular multiconnection networks of arbitrary size with a relatively simple control algorithm.     

不同池化模型的卷积神经网络学习性能研究

目的 基于卷积神经网络的深度学习算法在图像处理领域正引起广泛关注。为了进一步提高卷积神经网络特征提取的准确度,加快参数收敛速度,优化网络学习性能,通过对比不同的池化模型对学习性能的影响提出一种动态自适应的改进池化算法。方法 构建卷积神经网络模型,使用不同的池化模型对网络进行训练,并检验在不同迭代次数下的学习结果。在现有算法准确率不高和收敛速度较慢的情况下,通过使用不同的池化模型对网络进行训练,从而构建一种新的动态自适应池化模型,并研究在不同迭代次数下其对识别准确率和收敛速度的影响。结果 通过对比实验发现,使用动态自适应池化算法的卷积神经网络学习性能最优,在手写数字集上的收敛速度最高可以提升18.55%,而模型对图像的误识率最多可以降低20%。结论 动态自适应池化算法不但使卷积神经网络对特征的提取更加精确,而且很大程度地提高了收敛速度和模型准确率,从而达到优化网络学习性能的目的。这种模型可以进一步拓展到其他与卷积神经网络相关的深度学习算法。     

Multi-log2N交换网络的性能分析模型及控制算法

高速多平面交换网络解决了其内部冲突问题,但需要相应的路由控制算法的辅助,否则,内部冲突不能彻底解决.这是因为包在输入级路由平面的选择不够恰当,容易导致路由冲突的产生.因此,根据冲突链路集的思想,给出一种Multi-log₂N交换网络的控制算法.该算法控制分组在路由平面间的选择,不仅能够适用于RNB和SNB,还能实现单播和多播的控制,保障Multi-log₂N完全实现无阻塞.另一方面,Multi-log₂N消除了内部的链路冲突,提高了交换速率,但对其交换性能缺乏系统的理论分析.给出一种基于嵌入式马尔可夫链的分析模型,对Multi-log₂N网络中队列的使用及分组在队列中的平均等待时间、平均队长等相关性能指标进行了系统的分析,为基于Multi-log₂N的光交换节点的设计提供了良好的理论依据.