Global Convergence of GHA Learning Algorithm With Nonzero-Approaching Adaptive Learning Rates

The generalized Hebbian algorithm (GHA) is one of the most widely used principal component analysis (PCA) neural network (NN) learning algorithms. Learning rates of GHA play important roles in convergence of the algorithm for applications. Traditionally, the learning rates of GHA are required to converge to zero so that its convergence can be analyzed by studying the corresponding deterministic continuous-time (DCT) equations. However, the requirement for learning rates to approach zero is not a practical one in applications due to computational roundoff limitations and tracking requirements. In this paper, nonzero-approaching adaptive learning rates are proposed to overcome this problem. These proposed adaptive learning rates converge to some positive constants, which not only speed up the algorithm evolution considerably, but also guarantee global convergence of the GHA algorithm. The convergence is studied in detail by analyzing the corresponding deterministic discrete-time (DDT) equations. Extensive simulations are carried out to illustrate the theory.     

A unified learning algorithm to extract principal and minor components

Recently, many unified learning algorithms have been developed to solve the task of principal component analysis (PCA) and minor component analysis (MCA). These unified algorithms can be used to extract principal component and if altered simply by the sign, it can also serve as a minor component extractor. This is of practical significance in the implementations of algorithms. Convergence of the existing unified algorithms is guaranteed only under the condition that the learning rates of algorithms approach zero, which is impractical in many practical applications. In this paper, we propose a unified PCA & MCA algorithm with a constant learning rate, and derive the sufficient conditions to guarantee convergence via analyzing the discrete-time dynamics of the proposed algorithm. The achieved theoretical results lay a solid foundation for the applications of our proposed algorithm.     

A modified gradient-based neuro-fuzzy learning algorithm and its convergence

Neuro-fuzzy approach is known to provide an adaptive method to generate or tune fuzzy rules for fuzzy systems. In this paper, a modified gradient-based neuro-fuzzy learning algorithm is proposed for zero-order Takagi-Sugeno inference systems. This modified algorithm, compared with conventional gradient-based neuro-fuzzy learning algorithm, reduces the cost of calculating the gradient of the error function and improves the learning efficiency. Some weak and strong convergence results for this algorithm are proved, indicating that the gradient of the error function goes to zero and the fuzzy parameter sequence goes to a fixed value, respectively. A constant learning rate is used. Some conditions for the constant learning rate to guarantee the convergence are specified. Numerical examples are provided to support the theoretical findings.     

Adaptive multiple minor directions extraction in parallel using a PCA neural network

A principal component analysis (PCA) neural network is developed for online extraction of the multiple minor directions of an input signal. The neural network can extract the multiple minor directions in parallel by computing the principal directions of the transformed input signal so that the stability-speed problem of directly computing the minor directions can be avoided to a certain extent. On the other hand, the learning algorithms for updating the net weights use constant learning rates. This overcomes the shortcoming of the learning rates approaching zero. In addition, the proposed algorithms are globally convergent so that it is very simple to choose the initial values of the learning parameters. This paper presents the convergence analysis of the proposed algorithms by studying the corresponding deterministic discrete time (DDT) equations. Rigorous mathematical proof is given to prove the global convergence. The theoretical results are further confirmed via simulations.     

Forecasting foreign exchange rates with an improved back-propagation learning algorithm with adaptive smoothing momentum terms

The slow convergence of back-propagation neural network (BPNN) has become a challenge in data-mining and knowledge discovery applications due to the drawbacks of the gradient descent (GD) optimization method, which is widely adopted in BPNN learning. To solve this problem, some standard optimization techniques such as conjugate-gradient and Newton method have been proposed to improve the convergence rate of BP learning algorithm. This paper presents a heuristic method that adds an adaptive smoothing momentum term to original BP learning algorithm to speedup the convergence. In this improved BP learning algorithm, adaptive smoothing technique is used to adjust the momentums of weight updating formula automatically in terms of “3 σ limits theory.” Using the adaptive smoothing momentum terms, the improved BP learning algorithm can make the network training and convergence process faster, and the network’s generalization performance stronger than the standard BP learning algorithm can do. In order to verify the effectiveness of the proposed BP learning algorithm, three typical foreign exchange rates, British pound (GBP), Euro (EUR), and Japanese yen (JPY), are chosen as the forecasting targets for illustration purpose. Experimental results from homogeneous algorithm comparisons reveal that the proposed BP learning algorithm outperforms the other comparable BP algorithms in performance and convergence rate. Furthermore, empirical results from heterogeneous model comparisons also show the effectiveness of the proposed BP learning algorithm.     

Global convergence analysis of a discrete time nonnegative ICA algorithm

When the independent sources are known to be nonnegative and well-grounded, which means that they have a nonzero pdf in the region of zero, Oja and Plumbley have proposed a "Nonnegative principal component analysis (PCA)" algorithm to separate these positive sources. Generally, it is very difficult to prove the convergence of a discrete-time independent component analysis (ICA) learning algorithm. However, by using the skew-symmetry property of this discrete-time "Nonnegative PCA" algorithm, if the learning rate satisfies suitable condition, the global convergence of this discrete-time algorithm can be proven. Simulation results are employed to further illustrate the advantages of this theory.     

An adaptive learning algorithm for principal component analysis

Principal component analysis (PCA) is one of the most general purpose feature extraction methods. A variety of learning algorithms for PCA has been proposed. Many conventional algorithms, however, will either diverge or converge very slowly if learning rate parameters are not properly chosen. In this paper, an adaptive learning algorithm (ALA) for PCA is proposed. By adaptively selecting the learning rate parameters, we show that the m weight vectors in the ALA converge to the first m principle component vectors with almost the same rates. Comparing with the Sanger's generalized Hebbian algorithm (GHA), the ALA can quickly find the desired principal component vectors while the GHA fails to do so. Finally, simulation results are also included to illustrate the effectiveness of the ALA.     

Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm

The convergence of Oja's principal component analysis (PCA) learning algorithms is a difficult topic for direct study and analysis. Traditionally, the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time (DCT) systems. Such a method will require the learning rate to converge to zero, which is not a reasonable requirement to impose in many practical applications. Recently, deterministic discrete time (DDT) systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms. Unlike DCT systems, DDT systems allow learning rates to be constant (which can be a nonzero). This paper will provide some important results relating to the convergence of a DDT system of Oja's PCA learning algorithm. It has the following contributions: 1) A number of invariant sets are obtained, based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever. Thus, the nondivergence of the trajectories is guaranteed. 2) The convergence of the DDT system is analyzed rigorously. It is proven, in the paper, that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix. In addition, exponential convergence rate are obtained, providing useful guidelines for the selection of fast convergence learning rate. 3) Since the trajectories may diverge, the careful choice of initial vectors is an important issue. This paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence. 4) Simulation results will be furnished to illustrate the theoretical results achieved.     

Robust recursive least squares learning algorithm for principalcomponent analysis

A learning algorithm for the principal component analysis (PCA) is developed based on the least-square minimization. The dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components. The proposed algorithm is robust to the error accumulation existing in the sequential PCA algorithm. We show that all information needed for PCA can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input-output product. The updating of the normalized weight vector can be referred to as a leaky Hebb's rule. The convergence of the proposed algorithm is briefly analyzed. We also establish the relation between Oja's rule and the least squares learning rule. Finally, the simulation results are given to illustrate the effectiveness of this algorithm for PCA and tracking time-varying directions-of-arrival.     

基于多样性全局最优引导和反向学习的离子运动算法

针对离子运动算法空间探索能力和开发能力的不足,提出一种改进算法.在离子运动算法的液态阶段中,该算法嵌入一种多样性反馈搜索机制和全局最优引导策略的算法结构;同时,优化算法晶态阶段中的初始化过程采用反向学习方法生成,其中,初始化概率采用动态惯性改变方式.经过国际上通用的23个基准函数测试,与一些流行的元启发式算法比较,并从平均收敛值、方差、Wilcoxon符号秩检验、收敛成功率以及最优收敛时间等方面进行综合评估,从而表明所提出算法的有效性.     

Weighted and robust learning of subspace representations

A reliable system for visual learning and recognition should enable a selective treatment of individual parts of input data and should successfully deal with noise and occlusions. These requirements are not satisfactorily met when visual learning is approached by appearance-based modeling of objects and scenes using the traditional PCA approach. In this paper we extend standard PCA approach to overcome these shortcomings. We first present a weighted version of PCA, which, unlike the standard approach, considers individual pixels and images selectively, depending on the corresponding weights. Then we propose a robust PCA method for obtaining a consistent subspace representation in the presence of outlying pixels in the training images. The method is based on the EM algorithm for estimation of principal subspaces in the presence of missing data. We demonstrate the efficiency of the proposed methods in a number of experiments.     

Convergence analysis of the OJAn MCA learning algorithm by the deterministic discrete time method

Minor component analysis (MCA) is a statistical method of extracting the eigenvector associated with the smallest eigenvalue of the covariance matrix of input signals. Convergence is essential for MCA algorithms towards practical applications. Traditionally, the convergence of MCA algorithms is indirectly analyzed via their corresponding deterministic continuous time (DCT) systems. However, the DCT method requires the learning rate to approach zero, which is not reasonable in many applications due to the round-off limitation and tracking requirements. This paper studies the convergence of the deterministic discrete time (DDT) system associated with the OJAn MCA learning algorithm. Unlike the DCT method, the DDT method does not require the learning rate to approach zero. In this paper, some important convergence results are obtained for the OJAn MCA learning algorithm via the DDT method. Simulations are carried out to illustrate the theoretical results achieved.     

具有反向学习和自适应逃逸功能的粒子群优化算法

为克服粒子群优化算法进化后期收敛速度慢、易陷入局部最优等缺点,提出一种具有反向学习和自适应逃逸功能的粒子群优化算法.通过设定的阈值,算法将种群进化状态划分为正常状态和"早熟"状态: 若算法处于正常的进化状态,采用标准粒子群优化算法的进化模式;当粒子陷入"早熟"状态,运用反向学习和自适应逃逸功能,对个体最优位置进行反向学习,产生粒子的反向解,增加粒子的反向学习能力,增强算法逃离局部最优的能力,提高算法寻优率.在固定评估次数的情况下,对8个基准测试函数进行仿真,实验结果表明:所提算法在收敛速度、寻优精度和逃离局部最优的能力上明显优于多种经典粒子群优化算法,如充分联系的粒子群优化算法(FIPS)、基于时变加速度系数的自组织分层粒子群优化算法(HPSO-TVAC)、综合学习的粒子群优化算法(CLPSO)、自适应粒子群优化算法(APSO)、双中心粒子群优化算法(DCPSO)和具有快速收敛和自适应逃逸功能的粒子群优化算法(FAPSO)等.     

Supervised fuzzy reinforcement learning for robot navigation

This paper addresses a new method for combination of supervised learning and reinforcement learning (RL). Applying supervised learning in robot navigation encounters serious challenges such as inconsistent and noisy data, difficulty for gathering training data, and high error in training data. RL capabilities such as training only by one evaluation scalar signal, and high degree of exploration have encouraged researchers to use RL in robot navigation problem. However, RL algorithms are time consuming as well as suffer from high failure rate in the training phase. Here, we propose Supervised Fuzzy Sarsa Learning (SFSL) as a novel idea for utilizing advantages of both supervised and reinforcement learning algorithms. A zero order Takagi–Sugeno fuzzy controller with some candidate actions for each rule is considered as the main module of robot's controller. The aim of training is to find the best action for each fuzzy rule. In the first step, a human supervisor drives an E-puck robot within the environment and the training data are gathered. In the second step as a hard tuning, the training data are used for initializing the value (worth) of each candidate action in the fuzzy rules. Afterwards, the fuzzy Sarsa learning module, as a critic-only based fuzzy reinforcement learner, fine tunes the parameters of conclusion parts of the fuzzy controller online. The proposed algorithm is used for driving E-puck robot in the environment with obstacles. The experiment results show that the proposed approach decreases the learning time and the number of failures; also it improves the quality of the robot's motion in the testing environments.     

Grading learning for blind source separation

By generalizing the learning rate parameter to a learning rate matrix, this paper proposes a grading learning algorithm for blind source separation. The whole learning process is divided into three stages: initial stage, capturing stage and tracking stage. In different stages, different learning rates are used for each output component, which is determined by its dependency on other output components. It is shown that the grading learning algorithm is equivariant and can keep the separating matrix from becoming singular. Simulations show that the proposed algorithm can achieve faster convergence, better steady-state performance and higher numerical robustness, as compared with the existing algorithms using fixed, time-descending and adaptive learning rates.     

一种新的变学习速率自适应独立分量分析算法

学习速率的优选问题是自适应ICA算法中一个重要问题。论文建立了学习速率与相依性测度之间的一种非线性函数关系,以此为基础提出了一种新的变学习速率的自适应ICA算法,并且分析了参数a,b的取值原则及对算法收敛性能和稳态性能的影响。该算法能根据相依性测度所反映的信号分离的状态自适应地调节学习速率,克服了传统算法在稳态阶段步长调整过程中的不足。理论分析和计算机仿真结果都验证了算法的收敛性能和稳态性能。     

Effective gaussian mixture learning for video background subtraction

Adaptive Gaussian mixtures have been used for modeling nonstationary temporal distributions of pixels in video surveillance applications. However, a common problem for this approach is balancing between model convergence speed and stability. This paper proposes an effective scheme to improve the convergence rate without compromising model stability. This is achieved by replacing the global, static retention factor with an adaptive learning rate calculated for each Gaussian at every frame. Significant improvements are shown on both synthetic and real video data. Incorporating this algorithm into a statistical framework for background subtraction leads to an improved segmentation performance compared to a standard method.     

A composite energy function-based learning control approach for nonlinear systems with time-varying parametric uncertainties

A new learning control approach is developed in this note to address a class of nonlinear systems with time-varying parametric uncertainties. The concept of composite energy function (CEF), which provides the system information along both time and learning repetition horizons, is introduced in the analysis of learning control. CEF consists of two parts. The first part is a standard Lyapunov function,. which is used to access system behavior along time horizon during each learning cycle. The second part is an L/sup 2/ norm of parametric learning errors which reflects the variation of the system status when the control system is updated on the basis of learning cycles. The proposed learning control algorithm achieves asymptotical convergence along a learning repetition horizon. At the same time, the boundedness and pointwise convergence of the tracking error along time horizon is guaranteed. The proposed learning control strategy is applicable to quite general classes of nonlinear systems without requiring the global Lipschitz continuity condition and zero relative degree condition.     

基于AdaGrad的自适应NAG方法及其最优个体收敛性

与梯度下降法相比,自适应梯度下降方法(AdaGrad)利用过往平方梯度的算数平均保存了历史数据的几何信息,在处理稀疏数据时获得了更紧的收敛界.另一方面,Nesterov加速梯度方法(Nesterov's accelerated gradient,NAG)在梯度下降法的基础上添加了动量运算,在求解光滑凸优化问题时具有数量...     

Online adaptive algorithm for optimal control with integral reinforcement learning

In this paper, we introduce an online algorithm that uses integral reinforcement knowledge for learning the continuous‐time optimal control solution for nonlinear systems with infinite horizon costs and partial knowledge of the system dynamics. This algorithm is a data‐based approach to the solution of the Hamilton–Jacobi–Bellman equation, and it does not require explicit knowledge on the system's drift dynamics. A novel adaptive control algorithm is given that is based on policy iteration and implemented using an actor/critic structure having two adaptive approximator structures. Both actor and critic approximation networks are adapted simultaneously. A persistence of excitation condition is required to guarantee convergence of the critic to the actual optimal value function. Novel adaptive control tuning algorithms are given for both critic and actor networks, with extra terms in the actor tuning law being required to guarantee closed loop dynamical stability. The approximate convergence to the optimal controller is proven, and stability of the system is also guaranteed. Simulation examples support the theoretical result. Copyright © 2013 John Wiley & Sons, Ltd.