BP神经网络学习速度及泛化能力的提高

针对BP网络学习速度的缓慢性及较差的泛化能力,本文提出了几种改进的学习算法。其改进具体表现在以下方面:1)采用Cauchy误差估计器代替传统的LMS误差估计器;2 )对常规的Sigmoid函数引入形态因子;3)自适应修改学习率;4 )采用规则化调整提高泛化能力。并以MATLAB神经网络工具箱中相关函数验证了3) 4 )两种改进算法的优越性。     

基于函数集信息量的改进网络算法仿真研究

传统的神经网络学习算法往往存在欠学习或过学习情况,容易导致网络结构不够合理,预测函数泛化能力不理想.该文引用函数集信息量的概念,提出了以信噪比倒数为性能指标的改进型网络学习算法,深入分析了神经网络学习过程中欠学习和过学习的原因.经过仿真验证表明,该算法简单,自适应性强,收敛速度快,可以很好地克服欠学习和过学习问题,所得预测函数具有很好的泛化能力.     

神经网络泛化性能优化算法

基于提高神经网络泛化性能的目标提出了神经网络泛化损失率的概念,解析了与前一周期相比当前网络误差的变化趋势,在此基础上导出了基于泛化损失率的神经网络训练目标函数.利用新的目标函数和基于量子化粒子群算法的神经网络训练方法,得到了一种新的网络泛化性能优化算法.实验结果表明,将该算法与没有引入泛化损失率的算法相比,网络的收敛性能和泛化性能都有明显提高.     

激活函数可调的RBF神经网络模型

RBF神经网络是一种新颖的前馈神经网络,相比传统的BP神经网络具有更强的函数逼近、分类能力,以及更快的学习速度.在实际应用和科学研究中,不同径向基函数的选择对RBF神经网络的性能有很大的影响,一般是根据经验或者实验效果来选择适当的径向基函数.本文提出一种激活函数可调的RBF神经网络模型,将具有不同表达能力的径向基函数通过线性组合的形式形成新的神经元激活函数,进而提高神经网络的逼近精度和泛化能力,对于有监督学习,给出了相应的各参数的训练算法和应用举例.     

改进BP神经网络在木构古建筑中的寿命预测

标准的BP神经网络存在训练速度慢、容易陷入极小点、泛化能力低的特点。文中用附加动量项和改进学习速率相结合的方法对标准的BP神经网络进行了改进,并将其应用在木构古建筑的寿命预测中。仿真结果表明,和标准的BP神经网络相比,改进后的BP神经网络提高了泛化能力,能较准确地拟合训练值,避免了在确定计算参数过程中所产生的计算误差。     

一个基于模糊神经网络的数据逼近和泛化建模方法

本文介绍一个基于模糊神经网络的数据逼近和泛化建模方法,定义了一种模糊系统动态调节神经网络的学习率,给出了用迭代自组织数据分析算法确定神经网络结构、初始化神经网络参数的方法.在雷达天线罩视线误差建模中的应用表明,这种方法加快了网络的收敛速度,避免了局部极值,具有较高的数据逼近和泛化能力.     

基于复合适应度微粒群算法的神经网络训练

为提高神经网络的泛化能力,针对以均方误差为适应度的PSO算法在训练神经网络时会产生一定的过拟合问题,提出对均方误差和误差分布均匀度进行信息融合,构成复合适应度作为训练指标.实验结果表明,该方法可使网络的泛化能力得到明显的改善.     

改进的神经网络及其自适应学习速率的研究

从提高神经网络泛化能力的角度提出一种改进方法.利用Taylor级数展开的思想,用线性和非线性组合构成函数映射关系,即改进的神经网络是用原神经网络的非线性映射和关于输入信号的线性映射的和来逼近期望值.文中还给出了该神经网络学习速率的自适应调节方法.对线性对象和非线性对象分别进行建模仿真,结果表明,改进的神经网络比基于正则化方法的神经网络具有更好的泛化能力.     

B—P网络泛化性能的改善

在神经网络的训练过程中存在“过度吻合”的现象,即训练样本的误差已达到非常小的一个值,但是非训练样本的误差非常大,造成神经网络的泛化性能不好。本文说明了泛化性能与隐层节点数的关系,并提出了通过改变性能函数来改善B－P网络的泛化性能的方法。     

一种主动学习神经网络集成方法

分析了神经网络集成泛化误差、个体神经网络泛化误差、个体神经网络差异度之间的关系，提出了一种个体神经网络主动学习方法．个体神经网络同时交互训练，既满足了个体神经网络的精度要求，又满足了个体神经网络的差异性要求．另外，给出了一种个体神经网络选择性集成方法，对个体神经网络加入偏置量，增加了个体神经网络的可选数量，降低了神经网络集成的泛化误差．理论分析和实验结果表明，使用这种个体神经网络训练方法、个体神经网络选择性集成方法能够构建有效的神经网络集成系统．     

CMAC学习性能及泛化性能研究综述

小脑模型清晰度控制器(CMAC)是一种局部学习前馈网络，结构简单，收敛速度快，易于实现。从其每个神经元来看，各神经元之间是一种线性关系，但从总体结构来看，网络是一种非线性映射关系。而且模型从输入开始就存在一种泛化能力。网络的学习和泛化能力一直是研究热点，因此，该文将对CMAC网络的泛化能力、学习能力以及一些改善途径进行多方面的综合性的讨论。文章最后还将给出一种改善CMAC泛化能力的训练策略，它不仅避免了学习干扰问题加快了学习速度而且可以通过提高训练循环次数增加训练样本量。通过MATLAB仿真发现这种训练策略可以改善CMAC网络的泛化能力。该方法简单有效是可行的。     

Genetic programming and evolutionary generalization

In genetic programming (GP), learning problems can be classified broadly into two types: those using data sets, as in supervised learning, and those using an environment as a source of feedback. An increasing amount of research has concentrated on the robustness or generalization ability of the programs evolved using GP. While some of the researchers report on the brittleness of the solutions evolved, others proposed methods of promoting robustness/generalization. It is important that these methods are not ad hoc and are applicable to other experimental setups. In this paper, learning concepts from traditional machine learning and a brief review of research on generalization in GP are presented. The paper also identifies problems with brittleness of solutions produced by GP and suggests a method for promoting robustness/generalization of the solutions in simulating learning behaviors using GP     

基于Vapnik-Chervonenkis泛化界的极限学习机模型复杂性控制

模型复杂性是决定学习机器泛化性能的关键因素,对其进行合理的控制是模型选择的重要原则.极限学习机(extreme learning machine,ELM)作为一种新的机器学习算法,表现出了优越的学习性能.但对于如何在ELM的模型选择过程中合理地度量和控制其模型复杂性这一基本问题,目前尚欠缺系统的研究.本文讨论了基于Vapnik-Chervonenkis(VC)泛化界的ELM模型复杂性控制方法(记作VM),并与其他4种经典模型选择方法进行了系统的比较研究.在人工和实际数据集上的实验表明,与其他4种经典方法相比,VM具有更优的模型选择性能:能选出同时具有最低模型复杂性和最低(或近似最低)实际预测风险的ELM模型.此外,本文也为VC维理论的实际应用价值研究提供了一个新的例证.     

基于误差相关度学习样本选择

针对有限样本学习机器的偏差/方差的困境,以及过拟合引起的泛化性能的下降,分析了样本选择对学习机器泛化的影响,提出误差相关度学习算法ECL,利用误差相关度来权衡偏差和方差的关系,避免了求解复杂学习系统的VC维数,并以样本点的误差相关度为指标来选择训练子集,提高学习机器的泛化性能.仿真结果表明ECL算法有效地抑制过拟合现象的发生,保证学习机器泛化性能的提高.     

Stability-based PAC-Bayes analysis for multi-view learning algorithms

Multi-view learning exploits structural constraints among multiple views to effectively learn from data. Although it has made great methodological achievements in recent years, the current generalization theory is still insufficient to prove the merit of multi-view learning. This paper blends stability into multi-view PAC-Bayes analysis to explore the generalization performance and effectiveness of multi-view learning algorithms. We propose a novel view-consistency regularization to produce an informative prior that helps to obtain a stability-based multi-view bound. Furthermore, we derive an upper bound on the stability coefficient that is involved in the PAC-Bayes bound of multi-view regularization algorithms for the purpose of computation, taking the multi-view support vector machine as an example. Experiments provide strong evidence on the advantageous generalization bounds of multi-view learning over single-view learning. We also explore strengths and weaknesses of the proposed stability-based bound compared with previous non-stability multi-view bounds experimentally.     

并行效率敏感的大规模SVM数据分块数选择

数据分块数的选择是并行/分布式机器学习模型选择的基本问题之一,直接影响着机器学习算法的泛化性和运行效率。现有并行/分布式机器学习方法往往根据经验或处理器个数来选择数据分块数,没有明确的数据分块数选择准则。提出一个并行效率敏感的并行/分布式机器学习数据分块数选择准则,该准则可在保证并行/分布式机器学习模型测试精度的情况下,提高计算效率。首先推导并行/分布式机器学习模型的泛化误差与分块数目的关系。然后以此为基础,提出折衷泛化性与并行效率的数据分块数选择准则。最后,在ADMM框架下随机傅里叶特征空间中,给出采用该数据分块数选择准则的大规模支持向量机实现方案,并在高性能计算集群和大规模标准数据集上对所提出的数据分块数选择准则的有效性进行实验验证。     

基于DFP校正拟牛顿法的傅里叶神经网络

针对傅里叶神经网络采用最速下降法导致局部极小、学习速度慢以及泛化能力差的问题,提出一种基于DFP校正拟牛顿法的新学习算法。该算法计算复杂度低,能保证网络具有良好的泛化能力和全局最优性。通过2个数值算例检验该算法,同时和BP神经网络以及另外2种傅里叶神经网络作比较。结果表明,该算法计算复杂度约为最速下降法的5%,为最小二乘学习算法的80%,具有较好的泛化 能力。     

基于高斯回归的连续空间多智能体跟踪学习

提高适应性、实现连续空间的泛化、降低维度是实现多智能体强化学习（Multi-agent reinforcement learning,MARL）在连续系统中应用的几个关键. 针对上述需求,本文提出连续多智能体系统（Multi-agent systems,MAS）环境下基于模型的智能体跟踪式学习机制和算法（MAS MBRL-CPT）.以学习智能体适应同伴策略为出发点,通过定义个体期望即时回报,将智能体对同伴策略的观测融入环境交互效果中,并运用随机逼近实现个体期望即时回报的在线学习.定义降维的Q函数,在降低学习空间维度的同时,建立MAS环境下智能体跟踪式学习的Markov决策过程（Markov decision process,MDP）.在运用高斯回归建立状态转移概率模型的基础上,实现泛化样本集Q值函数的在线动态规划求解.基于离散样本集Q函数运用高斯回归建立值函数和策略的泛化模型. MAS MBRL-CPT在连续空间Multi-cart-pole控制系统的仿真实验表明,算法能够使学习智能体在系统动力学模型和同伴策略未知的条件下,实现适应性协作策略的学习,具有学习效率高、泛化能力强等特点.     

Generalization and multirate models of motor adaptation

When subjects adapt their reaching movements in the setting of a systematic force or visual perturbation, generalization of adaptation can be assessed psychophysically in two ways: by testing untrained locations in the work space at the end of adaptation (slow postadaptation generalization) or by determining the influence of an error on the next trial during adaptation (fast trial-by-trial generalization). These two measures of generalization have been widely used in psychophysical studies, but the reason that they might differ has not been addressed explicitly. Our goal was to develop a computational framework for determining when a two-state model is justified by the data and to explore the implications of these two types of generalization for neural representations of movements. We first investigated, for single-target learning, how well standard statistical model selection procedures can discriminate two-process models from single-process models when learning and retention coefficients were systematically varied. We then built a two-state model for multitarget learning and showed that if an adaptation process is indeed two-rate, then the postadaptation generalization approach primarily probes the slow process, whereas the trial-by-trial generalization approach is most informative about the fast process. The fast process, due to its strong sensitivity to trial error, contributes predominantly to trial-by-trial generalization, whereas the strong retention of the slow system contributes predominantly to postadaptation generalization. Thus, when adaptation can be shown to be two-rate, the two measures of generalization may probe different brain representations of movement direction.     

AL-ProMP: Force-relevant skills learning and generalization method for robotic polishing

Skill learning in robot polishing is gaining attention and becoming a hot issue. Current studies on skill learning in robot polishing are mainly about trajectory skills, and force-relevant skills learning models are less studied. A skill learning method with good generalization and robustness is one of the elements worth investigating. In this study, a force-relevant skills learning method called arc-length probabilistic movement primitives (AL-ProMP) is proposed to improve the efficiency of robot polishing force planning. AL-ProMP learns the mapping between the contact force and polishing trajectory, and the temporal scaling factor and force scaling factor in AL-ProMP enable better robustness of force planning in speed scaling tasks and polishing tasks in different scenarios. Speed scaling is an important property for adaptation of the polishing policy. For the generalization of polishing skills to different polishing tools in robotics disc polishing tasks of unknown geometric model workpieces, a novel force scaling factor for different polishing discs is proposed according to the contact force model. In addition, polishing contact position learning provides the basis for polishing trajectory generalization. Finally, it is experimentally verified that the proposed method is effective in learning and generalizing the demonstrated skills and improving the polishing surface quality of the workpiece with unknown geometric model.