基于罚函数内点法的泄露积分型回声状态网的参数优化

为了提升泄露积分型回声状态网（Leaky integrator echo state network，Leaky-ESN）的性能，提出利用罚函数内点法优化Leaky-ESN的全局参数，如泄漏率、内部连接权矩阵谱半径、输入比例因子等，这克服了通过反复试验法选取参数值而降低了Leaky-ESN模型的优越性和性能.Leaky-ESN的全局参数必须保障回声状态网满足回声状态特性，因此它们之间存在不等式约束条件.有学者提出利用随机梯度下降法来优化内部连接权矩阵谱半径、输入比例因子、泄露率三个全局参数，一定程度上提高了Leaky-ESN的逼近精度.然而，随机梯度下降法是解决无约束优化问题的基本算法，在利用随机梯度下降法优化参数时，没有考虑参数必须满足回声特性的约束条件（不等式约束条件），致使得到的参数值不是最优解.由于罚函数内点法可以求解具有不等式约束的最优化问题，应用范围广，收敛速度较快，具有很强的全局寻优能力.因此，本文提出利用罚函数内点法优化Leaky-ESN的全局参数，并以时间序列预测为例，检验优化后的Leaky-ESN的预测性能，仿真结果表明了本文提出方法的有效性.

Parameter Optimization of Leaky Integrator Echo State Network with Internal-point Penalty Function Method

To improve leaky integrator echo state network (Leaky-ESN) performance, internal-point penalty function (IPF) method is used to optimize the global parameters of Leaky-ESN, such as leakage rate, spectral radius of internal connection weight matrix, scaling of input, etc., which overcomes loss of superiority and performance of Leaky-ESN because of using trial and error method to select parameter values. The global parameters of Leaky-ESN have to guarantee the echo state network to meet the echo state property, thus inequality constraints exist between them. Some researchers put forward the method using the stochastic gradient descent (GD) to optimize leakage rate, spectral radius of internal connection weight matrix, and scaling of input, which can improve the approximation precision of the Leaky-ESN to some certain extent. However, the stochastic gradient descent method is a basic algorithm to solve unconstrained optimization problems. Without considering parameters which need satisfy the constraint conditions of the echo state property (inequality constraints) during using stochastic gradient descent method, the parameter value is not the optimal solution. Internal-point penalty function method can solve the optimized problem with inequality constraints, a wide scope of application, fast convergence speed, strong ability of global optimization. Therefore, in this paper, internal-point penalty function method is used to optimize the global parameters of Leaky-ESN, and time series prediction is selected as an example to examine the performance of the optimized Leaky-ESN. Simulation results show the effectiveness of the proposed approach.