基于混合搜索方向的前向复值神经网络的学习算法

复值有限内存BFGS(CL-BFGS)算法能有效用于求解复数域的无约束优化问题,但其性能容易受到记忆尺度的影响.为了解决记忆尺度的选择问题,提出一种基于混合搜索方向的CL-BFGS算法.对于给定的记忆尺度候选集,采用滑动窗口法将其划分成有限个子集,将各子集元素作为记忆尺度计算得到一组混合方向,选择使目标函数值最小的混合方向作为当前迭代的搜索方向.在迭代过程中,采用混合搜索方向的策略有益于强化对最新曲率信息的利用,便于记忆尺度的选取,提高算法的收敛速度,所提出的CL-BFGS算法适用于多层前向复值神经网络的高效学习.最后通过在模式识别、非线性信道均衡和复函数逼近上的实验验证了基于混合搜索方向的CLBFGS算法能取得比一些已有算法更好的性能.

Hybrid search direction based learning algorithm for feedforward complex-valued neural networks

Complex limited-memory BFGS (CL-BFGS) algorithm can be efficiently applied to solve unconstrained optimization problems in complex domain. However, its performance is seriously affected by memory size. In this paper, to deal with the selection problem of memory size, an improved CL-BFGS algorithm with hybrid search directions is proposed. The candidate set of memory size is divided into several parts by the sliding window method and a group of hybrid directions are constructed by considering the elements of each subset as potential memory sizes. Then the hybrid direction achieving the minimum value of objective function is taken as the actual search direction at the current iteration. The advantage of the strategy of hybrid search direction is to strengthen the usage of the latest curvature information and facilitate the choice of memory size such that the performance of the CL-BFGS algorithm is improved. The proposed CL-BFGS algorithm is then applied for the efficient learning of multi-layer feedforward complex-valued neural networks. Finally, experiments are conducted on the tasks of pattern recognition, nonlinear channel equalization and complex function approximation to verify that the proposed algorithm has better performance than some existing ones.