窗口长度自适应调整的策略迭代最优控制

在系统模型参数未知的最优控制问题中, 策略迭代能否快速收敛到最优控制策略的关键在于值函数的估计. 为了提升值函数的估计精度以及收敛速度, 本文提出一种窗口长度自适应调整的策略迭代最优控制算法. 充分利用一段时间内的历史样本数据, 通过影响力函数构建窗口长度与值函数估计性能之间的定量关系, 根据数据窗口长度对估计性能影响力的不同, 实现窗口长度的自适应调整. 最后, 将本文所提方法应用到连续发酵过程, 结果表明, 本文所提方法能够加快最优控制策略的收敛, 克服参数变化或外部扰动对控制性能的影响, 从而提升控制精度.

Optimal control of policy iteration with adaptive adjustment of window length

In the optimal control problem with unknown system model parameters, the key to whether the policy iteration can quickly converge to the optimal control policy is the estimation of the value function. In order to improve the estimation accuracy and speed of the value function, this paper proposes a policy iteration optimal control algorithm with adaptive window length adjustment. By making full use of the historical sample data within a period of time, the influence function is used to construct the quantitative relationship between the window length and the estimation performance of the value function, and the window length is adaptively adjusted according to the different influence of the data window length on the estimation performance. Finally, the proposed method is applied to the continuous fermentation process. Simulation results show that the proposed method can accelerate the convergence of the optimal control policy, overcome the influence of parameter changes or external disturbances on the control performance, and improve the control accuracy.