Global Random Optimization by Simultaneous Perturbation Stochastic Approximation

We examine the theoretical and numerical global convergence properties of a certain ldquogradient freerdquo stochastic approximation algorithm called the ldquosimultaneous perturbation stochastic approximation (SPSA)rdquo that has performed well in complex optimization problems. We establish two theorems on the global convergence of SPSA, the first involving the well-known method of injected noise. The second theorem establishes conditions under which ldquobasicrdquo SPSA without injected noise can achieve convergence in probability to a global optimum, a result with important practical benefits.     

基于类间最近邻支持向量信息测度排序的快速分类算法研究

提出了基于特征空间中最近邻类间支持向量信息测度排序的快速支持向量机分类算法,对于训练样本首先进行最近邻类间支持向量信息测度升序排列处理;然后根据排序的结果选择最优的训练样本子空间,在选择的样本子空间内采用乘性规则直接求取Lagrange因子,而不是传统的二次优化方法;最后加入附加剩余样本进行交叉验证处理,直到算法满足收敛性准则。各种分类实验结果表明,该算法具有非常良好的性能,特别是在训练样本庞大,支持向量数量较多的情况下,能够较大幅度地减少计算复杂度,提高分类速度。     

Actor-critic algorithms for hierarchical Markov decision processes

We consider the problem of control of hierarchical Markov decision processes and develop a simulation based two-timescale actor-critic algorithm in a general framework. We also develop certain approximation algorithms that require less computation and satisfy a performance bound. One of the approximation algorithms is a three-timescale actor-critic algorithm while the other is a two-timescale algorithm, however, which operates in two separate stages. All our algorithms recursively update randomized policies using the simultaneous perturbation stochastic approximation (SPSA) methodology. We briefly present the convergence analysis of our algorithms. We then present numerical experiments on a problem of production planning in semiconductor fabs on which we compare the performance of all algorithms together with policy iteration. Algorithms based on certain Hadamard matrix based deterministic perturbations are found to show the best results.     

一种分阶梯度加权平均变步长VLMP算法

通过平滑梯度矢量减小梯度估计误差,采用平滑梯度矢量的欧氏范数和误差信号的分数低阶矩更新步长因子,对一阶和二阶权系数采取分阶迭代更新,得到一种在[α]稳定分布噪声背景下变步长Volterra自适应滤波算法,分析证明了该算法的收敛性能。非线性系统辨识的仿真结果表明,算法较DOVLMP算法具有更快的收敛速度和更小的稳态失调。     

A diagonal recurrent neural network-based hybrid direct adaptiveSPSA control system

A direct adaptive simultaneous perturbation stochastic approximation (DA SPSA) control system with a diagonal recurrent neural network (DRNN) controller is proposed. The DA SPSA control system with DRNN has simpler architecture and parameter vector size that is smaller than a feedforward neural network (FNN) controller. The simulation results show that it has a faster convergence rate than FNN controller. It results in a steady-state error and is sensitive to SPSA coefficients and termination condition. For trajectory control purpose, a hybrid control system scheme with a conventional PID controller is proposed     

Vibration suppression control of a flexible arm using a nonlinear observer with simultaneous perturbation stochastic approximation

The main objective in this study is the vibrational control of a one-link flexible arm system. A variable structure system (VSS) nonlinear observer has been proposed in order to reduce the oscillation when controlling the angle-of the flexible arm. The parameters of the nonlinear observer are optimized using a modified version of the simultaneous perturbation stochastic approximation (SPSA) algorithm. The SPSA algorithm is especially useful when the number of parameters to be adjusted is large, and makes it possible to estimate them simultaneously. For the vibration and position control, a model reference sliding-mode control (MR-SMC) has been proposed. The MR-SMC parameters are also optimized using a modified version of the SPSA algorithm. The simulations show that vibrational control of a one-link flexible arm system can be achieved more efficiently using our method. Therefore, by applying the MR-SMC method to a nonlinear observer, we can improve the performance in this kind of model using our proposed SPSA algorithm, and we can determine the control parameters very easy and efficiently.     

The landscape adaptive particle swarm optimizer

Several modified particle swarm optimizers are proposed in this paper. In DVPSO, a distribution vector is used in the update of velocity. This vector is adjusted automatically according to the distribution of particles in each dimension. In COPSO, the probabilistic use of a ‘crossing over’ update is introduced to escape from local minima. The landscape adaptive particle swarm optimizer (LAPSO) combines these two schemes with the aim of achieving more robust and efficient search. Empirical performance comparisons between these new modified PSO methods, and also the inertia weight PSO (IFPSO), the constriction factor PSO (CFPSO) and a covariance matrix adaptation evolution strategy (CMAES) are presented on several benchmark problems. All the experimental results show that LAPSO is an efficient method to escape from convergence to local optima and approaches the global optimum rapidly on the problems used.     

Optimal random perturbations for stochastic approximation using asimultaneous perturbation gradient approximation

The simultaneous perturbation stochastic approximation (SPSA) algorithm has attracted considerable attention for challenging optimization problems where it is difficult or impossible to obtain a direct gradient of the objective (say, loss) function. The approach is based on a highly efficient simultaneous perturbation approximation to the gradient based on loss function measurements. SPSA is based on picking a simultaneous perturbation (random) vector in a Monte Carlo fashion as part of generating the approximation to the gradient. This paper derives the optimal distribution for the Monte Carlo process. The objective is to minimize the mean square error of the estimate. The authors also consider maximization of the likelihood that the estimate be confined within a bounded symmetric region of the true parameter. The optimal distribution for the components of the simultaneous perturbation vector is found to be a symmetric Bernoulli in both cases. The authors end the paper with a numerical study related to the area of experiment design     

基于相关函数的改进LMS自适应滤波算法研究

标准LMS算法由于采用了固定的步长因子,致使其算法稳态误差大、实时性差,不适用于非平稳随机过程.提出了一种改进的CLMS变步长自适应滤波算法,该算法采用输入-误差信号的互相关函数控制步长更新,在算法收敛初期步长取较大值,使其有较快的实时跟踪能力,在算法收敛末期步长取较小值,以使其拥有较小的稳态误差.仿真结果表明,相比已有变步长算法,CLMS算法有更好的应用性能.     

Iterative Learning Control for Nonlinear Systems: A Bounded‐Error Algorithm

This paper presents a nonlinear iterative learning control (NILC) for nonlinear time‐varying systems. An algorithm of a new strategy for the NILC implementation is proposed. This algorithm ensures that trajectory‐tracking errors of the proposed NILC, when implemented, are bounded by a given error norm bound. A special feature of the algorithm is that the trial‐time interval is finite but not fixed as it is for the other iterative learning algorithms. A sufficient condition for convergence and robustness of the bounded‐error learning procedure is derived. With respect to the bounded‐error and standard learning processes applied to a virtual robot, simulation results are presented in order to verify maximal tracking errors, convergence and applicability of the proposed learning control.     

量子门线路神经网络及其改进学习算法研究

量子门线路神经网络（QGCNN）是一种直接利用量子理论设计神经网络拓扑结构或训练算法的量子神经网络模型。动量更新是在神经网络的权值更新中加入动量,在改变权值向量的同时提供一个特定的惯量,从而避免权值向量在网络训练过程中持续振荡。在基本的量子门线路神经网络的学习算法中引入动量更新原理,提出了一种具有动量更新的量子门线路网络算法（QGCMA）。研究表明,QGCMA保持了网络100%的收敛率,同时,相对于基本算法,在具有相同学习速率的情况下,提高了网络的收敛速度。     

A Rayleigh weighted variable step-size sign algorithm for robust adaptive filtering in impulsive noises

The sign algorithm with a fixed step-size is incapable of addressing the conflicting requirements between fast convergence speed and low steady-state misadjustments. In order to deal with this problem, a Rayleigh weighted gradient vector based variable step-size sign algorithm is proposed in this paper. In the new algorithm, the variable step-size is updated by the squared norm of a Rayleigh weighted sign gradient vector. The proposed algorithm can improve the convergence speed and tracking capability while maintaining the similar steady-state misadjustments in the presence of impulsive noises. A complex-valued energy conservation relation based convergence analysis is carried out to evaluate the convergence performance of the new algorithm. Simulation results are presented to verify the theoretical analysis and to demonstrate the desirable performance of the proposed algorithm.     

Robust Adaptive Identification of Nonlinear System Using Neural Network

It is well known that disturbance can cause divergence of neural networks in the identification of nonlinear systems. Sufficient conditions using so‐called modified algorithms are available to provide guaranteed convergence for adaptive system. They are dead zone scheme, adaptive law modification, and σ‐modification. These schemes normally require knowledge of the upper bound of the disturbance. In this paper, a robust weighttuning algorithm is used to train the multi‐layered neural network with an adaptive dead zone scheme. The proposed robust adaptive algorithm does not require knowledge of either the upper bound of the disturbance or the bound on the norm of the estimate parameter. A complete convergence proof is provided based on Lyapunov theorem to deal with the nonlinear system. Simulation results are presented to show good perfor‐mance of the algorithm.     

基于向量整体扰动的快速收敛人工蜂群算法

针对标准人工蜂群算法收敛速度慢和易陷入早熟收敛等问题,提出一种快速收敛人工蜂群算法。首先借助反向学习理论初始化种群来提高初始解的分布质量,并在雇佣蜂和跟随蜂阶段引入向量整体扰动搜索方程加快局部搜索;然后为了跳出局部最优,采用一种随机更新搜索策略来增加蜂群多样性以平衡全局探索和局部利用能力;最后通过八个标准测试函数的仿真实验,发现所提出的算法与几个改进的人工蜂群算法相比,具有更快的收敛速度且获得了更高的求解精度,验证了算法的优越性。     

基于函数逼近的冗余值迭代算法*

针对值迭代算法存在算法收敛不稳定及收敛速度慢的问题,文中提出改进的基于函数逼近的冗余值迭代算法.结合值迭代算法与贝尔曼冗余值迭代算法,引入权重因子,构建值函数参数更新向量.同时从理论上证明,利用此更新向量更新值函数参数可以保证算法收敛,解决值迭代算法收敛不稳定的问题.此外,算法引入遗忘因子,加快权重向量的更新速率和算法收敛速度.在Grid World问题上的实验表明,文中算法收敛性能较好,具有较好的鲁棒性.     

Combined inverse and gradient iterative learning control: performance,monotonicity, robustness and non‐minimum‐phase zeros

Based on recent papers that have demonstrated that robust iterative learning control can be based on parameter optimization using either the inverse plant or gradient concepts, this paper presents a unification of these ideas for discrete‐time systems that not only retains the convergence properties and the robustness properties derived in previous papers but also permits the inclusion of filters in the input update formula and a detailed analysis of the effect of non‐minimum‐phase dynamics on algorithm performance in terms of a ‘plateauing’ or ‘flat‐lining’ effect in the error norm evolution. Although the analysis is in the time domain, the robustness conditions are expressed as frequency domain inequalities. The special case of a version of the inverse algorithm that can be used to construct a robust stable anti‐causal inverse non‐minimum‐phase plant is presented and analysed in detail. Copyright © 2012 John Wiley & Sons, Ltd.     

A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks

Training of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for online applications. Conventional RNN training algorithms such as the backpropagation through time and real-time recurrent learning have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes three specially designed adaptive parameters to maximize training speed for a recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.     

Comparative study of stochastic algorithms for system optimizationbased on gradient approximations

Stochastic approximation (SA) algorithms can be used in system optimization problems for which only noisy measurements of the system are available and the gradient of the loss function is not. This type of problem can be found in adaptive control, neural network training, experimental design, stochastic optimization, and many other areas. This paper studies three types of SA algorithms in a multivariate Kiefer-Wolfowitz setting, which uses only noisy measurements of the loss function (i.e., no loss function gradient measurements). The algorithms considered are: the standard finite-difference SA (FDSA) and two accelerated algorithms, the random directions SA (RDSA) and the simultaneous-perturbation SA (SPSA). RDSA and SPSA use randomized gradient approximations based on (generally) far fewer function measurements than FDSA in each Iteration. This paper describes the asymptotic error distribution for a class of RDSA algorithms, and compares the RDSA, SPSA, and FDSA algorithms theoretically (using mean-square errors computed from asymptotic distributions) and numerically. Based on the theoretical and numerical results, SPSA is the preferable algorithm to use.     

求解最大割问题的交叉熵算法

交叉熵方法（Cross Entropy）是近几年发展而来的一种启发式方法,在求解组合优化问题中显示出其简单有效的特点,将运用交叉熵方法（CE）寻求图论中一个典型的NP困难问题—最大割问题的最优解。为了解决最大割问题,CE方法借助Bernoulli分布的思想,将一个确定性的网络转换成一个具有一定随机性的关联网络,接下来首先按照一个多维的Bernoulli概率分布生成样本,同时计算出随机割;其次,基于前一步的数据,更新Bernoulli概率分布P参数,使得分布参数逐步逼近最优值产生最大割的稳定估计值。数值实验表明,CE方法具有很好的稳定性和收敛性,最终也获得了比较好的近似解。     

一种分级竞争紧致遗传算法的研究

根据随机游动理论,研究了二进制编码的紧致遗传算法中概率向量的进化特性,提出了一种分级竞争模式紧致遗传算法(GCGA),该算法加大参与竞争的两个个体的适应度的差距,目的在于使概率向量有效进化。在数值函数优化问题中进行仿真实验,结果表明,分级竞争模式紧致遗传算法收敛速度更快,全局寻优能力也得到提高。