Obtaining initial parameter estimates for nonlinear systems using multicriteria associative memories

Parameter estimation problems for nonlinear systems are typically formulated as nonlinear optimization problems. For such problems, one has the usual difficulty that standard successive approximation schemes require good initial estimates for the parameter vector. This paper develops a simple multicriteria associative memory (MAM) procedure for obtaining useful initial parameter estimates for nonlinear systems. An easily calculated one-parameter family of associative memory matrices is developed; see Equation (25). Each memory matrix is efficient with respect to two criteria: accurate recovery of parameter-output training case associations; and small matrix norm to guard against noise arising from imprecise calculations and observations. For illustration, the MAM procedure is used to obtain initial parameter estimates for a well-known nonlinear economic model, the Solow-Swan growth model. Surprisingly accurate initial parameter estimates are obtained over broad ranges of the family of MAM memory matrices, even when observations are corrupted by i.i.d. or correlated noise.     

Unidirectional loop network layout by a LP heuristic and design of telecommunications networks

In this paper, we present a new linear programming-based heuristic procedure for optimal design of the unidirectional loop network layout problem. The heuristic procedure employs a linear programming formulation and solves the problem using the flow matrix of the unidirectional loop problem. To find an optimal solution, one can either generate all possible solutions or use a branch-and-bound procedure. But, both above methods require very high computational time and computer memory for larger problems. The heuristic developed in this paper is quite fast and obtains near optimal solutions. The heuristic procedure was tested on 16 different problems selected from the literature. The results showed that in most cases optimal—and in a few cases near optimal—solutions were obtained with very little computational time. Several examples are discussed. We also demonstrate that the above problem formulation and approach can be used to solve a special class of telecommunication networks where a set of computers (or processors) are attached by unidirectional point-to-point links around a loop.     

基于线性规划的联想记忆神经网络模型

提出一种基于优化线性函数的神经网络联想记忆器,打破了将待识别模式作为网络起始点的常规,它能保证渐近稳定的平衡点集与样本点集相同,吸引域分布合理,不渐近稳定的平衡点恰为实际的拒识模式,并且电路实现容易,对拒识模式有清楚的解释。理论分析和计算机模拟都表明本文的模型是理想的联想记忆器,还可降低对硬件的精度要求。     

具有非线性映射能力的高性能联想记忆模型

通常的联想记忆模型的联想性能由于受到输入模式间交叉相关项的影响而有所下降，并且在输入与输出之间缺乏非线性映射能力。本文介绍一种高性能联想记忆模型，它将低维输入向量映射到一个高维的中间向量，从而提高了系统的联想能力，又使系统具有非线性映射能力，最后给出了几种推广。     

Nonlinear parameter estimation by weighted linear associativememory with nonzero interception

The method of linear associative memory (LAM) has recently been applied in nonlinear parameter estimation. In the method of LAM, a model response, nonlinear with respect to the parameters, is approximated linearly by a matrix, which maps inversely from a response vector to a parameter vector. This matrix is determined from a set of initial training parameter vectors and their response vectors according to a given cost function, and can be updated recursively and adaptively with a pair of newly generated parameter-response vector. The advantage of LAM is that it can yield good estimation of the true parameter from a given observed response even if the initial training parameter vectors are far from the true values. In a previous paper, we have significantly improved the LAM method by introducing a weighted linear associative memory (WLAM) approach for nonlinear parameter estimation. In the WLAM approach, the contribution of each pair of parameter-response vector to the cost function is weighted in a way such that if a response vector is closer to the observed one then its pair plays more important role in the cost function. However, in both LAM and WLAM, the linear association is introduced with zero interceptions, which would not give an exact association even if the model function is linear and so will affect the efficiency of the estimations. In this paper, we construct a theory which introduces a linear association memory with a nonzero interception (WLAMB). The results of our estimation tests on two quite different models, Van der Pol equation and somatic shunt cable model, suggest that WLAMB can still significantly improve on WLAM.     

Exponential stability analysis for neutral BAM neural networks with time-varying delays and stochastic disturbances

This paper is concerned with the global exponential stability analysis problem for a class of neutral bidirectional associative memory (BAM) neural networks with time-varying delays and stochastic dist...     

Immunity-based hybrid learning methods for approximator structure and parameter adjustment

From the point of view of information processing the immune system is a highly parallel and distributed intelligent system which has learning, memory, and associative retrieval capabilities. In this paper we present two immunity-based hybrid learning approaches for function approximation (or regression) problems that involve adjusting the structure and parameters of spatially localized models (e.g., radial basis function networks). The number and centers of the receptive fields for local models are specified by immunity-based structure adaptation algorithms, while the parameters of the local models, which enter in a linear fashion, are tuned separately using a least-squares method. The effectiveness of the procedure is demonstrated through a nonlinear function approximation problem and a nonlinear dynamical system modeling problem.     

混沌神经网络的研究进展

回顾了近年来几种主要混沌神经元模型及混沌神经网络的研究进展，介绍了其特点及主要的应用。已有的研究结果表明，混沌神经网络在求解复杂优化问题和联想记忆等方面比现有网络有着更好的性能。     

Financial distress prediction using the hybrid associative memory with translation

This paper presents an alternative technique for financial distress prediction systems. The method is based on a type of neural network, which is called hybrid associative memory with translation. While many different neural network architectures have successfully been used to predict credit risk and corporate failure, the power of associative memories for financial decision-making has not been explored in any depth as yet. The performance of the hybrid associative memory with translation is compared to four traditional neural networks, a support vector machine and a logistic regression model in terms of their prediction capabilities. The experimental results over nine real-life data sets show that the associative memory here proposed constitutes an appropriate solution for bankruptcy and credit risk prediction, performing significantly better than the rest of models under class imbalance and data overlapping conditions in terms of the true positive rate and the geometric mean of true positive and true negative rates.     

动态进化多目标优化中的串式记忆方法

如何利用过去搜索到的最优解对新的环境变化做出快速响应,是动态进化多目标优化(Dynamic Evolutio-nary Multi- objective Optimization,DEMO)研究的一大挑战。为此提出了一种串式记忆(Bunchy Memory,BM)方法。设计了基于极小化效应函数的抽取过程,从非支配集中抽取一串记忆串,以便保持记忆的多样性;将记忆体组织成串式队列的方式,以便将过去数次环境变化下抽取的记忆串存入记忆体;提出了基于二进制锦标赛选择的检索过程以复用记忆体中过去的最优解,来快速响应新的变化。BM方法具有良好的记忆效果,显著地提高了DEMO算法的收敛性和多样性。4个标准测试问题上的实验结果表明,BM方法比其它3种方法具有更好的记忆能力。相应地,集成了BM方法的DEMO算法所获得解集的收敛性与多样性也明显好于其它3种DEMO算法。     

Fixed-order FIR approximation of linear systems from quantized input and output data

The problem of identifying a fixed-order FIR approximation of linear systems with unknown structure, assuming that both input and output measurements are subjected to quantization, is dealt with in this paper. A fixed-order FIR model providing the best approximation of the input–output relationship is sought by minimizing the worst-case distance between the output of the true system and the modeled output, for all possible values of the input and output data consistent with their quantized measurements. The considered problem is firstly formulated in terms of robust optimization. Then, two different algorithms to compute the optimum of the formulated problem by means of linear programming techniques are presented. The effectiveness of the proposed approach is illustrated by means of a simulation example.     

On the problem of spurious patterns in neural associative memorymodels

The problem of spurious patterns in neural associative memory models is discussed. Some suggestions to solve this problem from the literature are reviewed and their inadequacies are pointed out. A solution based on the notion of neural self-interaction with a suitably chosen magnitude is presented for the Hebbian learning rule. For an optimal learning rule based on linear programming, asymmetric dilution of synaptic connections is presented as another solution to the problem of spurious patterns. With varying percentages of asymmetric dilution it is demonstrated numerically that this optimal learning rule leads to near total suppression of spurious patterns. For practical usage of neural associative memory networks a combination of the two solutions with the optimal learning rule is recommended to be the best proposition.     

Matrix Pseudoinversion in the Problem of Design of Associative Memory

A new method of design of associative memory (AM) is considered. The method is based on general solutions of linear systems. New relations for the direct and inverse Greville formulas are proposed to recurrently solve the problem of synthesis of AM with the properties of storing and forgetting.     

约束最小平方联想存储器及性能比较

鉴于Ｋｏｈｏｎｅｎ的最佳联想存储器对带噪输入会产生难以接受的联想误差，文中试图通过在Ｋｏｈｏｎｅｎ模型中引入对连接权阵的某种约束并进而优化，使修改后的Ｋｏｈｏｎｅｎ模型（ＣＬＳＡＭ）对带噪输入具有最小误差的联想．借助奇异值分解（ＳＶＤ）理论的分析和计算机模拟证实了ＣＬＳＡＭ的性能优越性．     

异构无线传感器网络中多目标优化节点部署策略

针对异构无线传感器网络节点高密度部署和事件发生存在"热点区域"问题,以区域覆盖率最大和网络能耗最小为优化目标,提出了一种基于多目标优化的二进制粒子群算法,对节点部署进行多目标优化。该算法采用概率感知模型,引入强支配系数使得解分布均匀,结合Pareto最优解选择排序和基于自适应权重的适应度分配,进而获得异构节点部署解。仿真结果表明:该算法能对目标空间进行广泛搜索,与NSGA—Ⅱ算法相比,算法具有良好的收敛性,能有效地提高网络的覆盖率和降低网络能耗。     

Stability and Hopf Bifurcation in a Simplified BAM Neural Network With Two Time Delays

Various local periodic solutions may represent different classes of storage patterns or memory patterns, and arise from the different equilibrium points of neural networks (NNs) by applying Hopf bifurcation technique. In this paper, a bidirectional associative memory NN with four neurons and multiple delays is considered. By applying the normal form theory and the center manifold theorem, analysis of its linear stability and Hopf bifurcation is performed. An algorithm is worked out for determining the direction and stability of the bifurcated periodic solutions. Numerical simulation results supporting the theoretical analysis are also given     

New delay-dependent exponential stability criteria of BAM neural networks with time delays

In this paper, the global exponential stability is investigated for the bi-directional associative memory networks with time delays. Several new sufficient conditions are presented to ensure global exponential stability of delayed bi-directional associative memory neural networks based on the Lyapunov functional method as well as linear matrix inequality technique. To the best of our knowledge, few reports about such “linearization” approach to exponential stability analysis for delayed neural network models have been presented in literature. The method, called parameterized first-order model transformation, is used to transform neural networks. The obtained conditions show to be less conservative and restrictive than that reported in the literature. Two numerical simulations are also given to illustrate the efficiency of our result.     

Passivity analysis for uncertain BAM neural networks with time delays and reaction–diffusions

This article deals with the problem of passivity analysis for delayed reaction–diffusion bidirectional associative memory (BAM) neural networks with weight uncertainties. By using a new integral inequality, we first present a passivity condition for the nominal networks, and then extend the result to the case with linear fractional weight uncertainties. The proposed conditions are expressed in terms of linear matrix inequalities, and thus can be checked easily. Examples are provided to demonstrate the effectiveness of the proposed results.     

Introducing robustness in multi-objective optimization

In optimization studies including multi-objective optimization, the main focus is placed on finding the global optimum or global Pareto-optimal solutions, representing the best possible objective values. However, in practice, users may not always be interested in finding the so-called global best solutions, particularly when these solutions are quite sensitive to the variable perturbations which cannot be avoided in practice. In such cases, practitioners are interested in finding the robust solutions which are less sensitive to small perturbations in variables. Although robust optimization is dealt with in detail in single-objective evolutionary optimization studies, in this paper, we present two different robust multi-objective optimization procedures, where the emphasis is to find a robust frontier, instead of the global Pareto-optimal frontier in a problem. The first procedure is a straightforward extension of a technique used for single-objective optimization and the second procedure is a more practical approach enabling a user to set the extent of robustness desired in a problem. To demonstrate the differences between global and robust multi-objective optimization principles and the differences between the two robust optimization procedures suggested here, we develop a number of constrained and unconstrained test problems having two and three objectives and show simulation results using an evolutionary multi-objective optimization (EMO) algorithm. Finally, we also apply both robust optimization methodologies to an engineering design problem.