基于光滑化$L_1$正则项的随机配置网络

为了提高随机配置网络(stochastic configuration networks,SCN)的泛化能力,提出一种适用于SCN的光滑化$L_1$正则化方法.针对$L_1$正则化算子局部不可微的缺陷,在曲线不光滑点的邻域内进行光滑处理,并在此基础上构建SCN的光滑误差函数,提出增量计算权值的算法,进而以交替方向乘子法为基础给出权值的全局优化算法,并且在理论上分析算法的收敛性.与$L_1$正则化的稀疏性和$L_2$正则化均匀减小参数的特点相比,所提出方法按重要程度保留数据的全部特征,使参数既保持在较小的范围内又具有层次分明的分布,从而使网络具有更好的泛化能力.最后,通过数值仿真实验验证了所提出方法的可行性和有效性.

Smoothing $L_1$ regularization for stochastic configuration networks

In order to improve the generalization capability of stochastic configuration networks (SCNs), a smooth $L_1$ regularization method for SCNs is proposed. Aiming at the defect of local non-differentiability of the $L_1$ regularization operator, smoothing is carried out in the neighborhood of non-smooth points of the curve. The convex error function of the SCN is constructed on this basis, and an algorithm for incremental calculation of the weights of the SCN is proposed. Furthermore, the global optimization algorithm is proposed based on the alternating direction multiplier method, and the convergence of the algorithm is analyzed theoretically. Compared with the sparsity of $L_1$ regularization and the uniform reduction of parameters by $L_2$ regularization, the proposed method retains all features of the data according to the degree of importance, the parameters are not only kept in a small range, but also have hierarchical distribution, so that the network has better generalization ability. Finally, the feasibility and effectiveness of the proposed method are verified by some numerical simulations.