求解大规模问题的多核学习正则化路径算法

多核学习在处理异构、不规则和分布不平坦的样本数据时表现出良好的灵活性和可解释性.针对精确正则化路径算法难以处理大规模样本数据的问题,文中提出正则化路径近似算法.根据采样分布函数进行抽样,在原始核矩阵的基础上生成近似矩阵,同时在拉格朗日乘子向量中抽取对应行,实现矩阵乘积的近似计算,提高多核学习正则化路径的求解效率.最后分析多核学习正则化路径近似算法的近似误差界和计算复杂性.在标准数据集上的实验验证文中算法的合理性和计算效率.     

分阶Dyna强化学习算法

针对现有Dyna强化学习体系结构下,"规划"和"学习"在计算资源分配上的不合理问题,提出了一种分阶Dyna体系结构,随着经验知识的不断积累,将学习过程划分为探索阶段、变比重学习阶段和优化阶段,分别进行"规划"和"学习"的协调控制,大大减少了计算资源的浪费.结合传统的Q-学习算法,设计了分阶Dyna-Q强化学习算法,以适应动态不确定环境下的任务.在一个标准强化学习问题中,验证了所设计的分阶Dyna强化学习算法比基本Dyna强化学习算法具有更好的学习性能.     

平均报酬模型的多步强化学习算法

讨论模型未知的平均报酬强化学习算法。通过结合即时差分学习与Ｒ学习算法,将折扣问题中的一些方法推广到了平均准则问题中,提出了两类算法：Ｒ（λ）学习。现有的Ｒ学习可视为Ｒ（λ）学习和ＴＴＤ（λ）学习当λ＝０时的一个特例。仿真结果表明,λ取中间值的Ｒ（λ）和ＴＴＤ（λ）学习比现有的方法在可靠性与收敛速度上均有提高。     

一种加速时间差分算法收敛的方法

时间差分算法(Temporal difference methods,TD)是一类模型无关的强化学习算法.该算法拥有较低的方差和可以在线(On-line)学习的优点,得到了广泛的应用.但对于一种给定的TD算法,往往只能通过调整步长参数或其他超参数来加速收敛,这也就造成了加速TD算法收敛的方法匮乏.针对此问题提出了一种利...     

平均奖赏强化学习算法研究

顺序决策问题常用马尔可夫决策过程(MDP)建模.当决策行为执行从时刻点扩展到连续时间上时,经典的马尔可夫决策过程模型也扩展到半马尔可夫决策过程模型(SMDP).当系统参数未知时,强化学习技术被用来学习最优策略.文中基于性能势理论,证明了平均奖赏强化学习的逼近定理.通过逼近相对参考状态的性能势值函数,研究一个新的平均奖赏强化学习算法--G-学习算法.G-学习算法既可以用于MDP,也可以用于SMDP.不同于经典的R-学习算法,G-学习算法采用相对参考状态的性能势值函数替代相对平均奖赏和的相对值函数.在顾客访问控制和生产库存仿真实验中,G-学习算法表现出优于R-学习算法和SMART算法的性能.     

平均报酬模型强化学习理论、算法及应用

折扣报酬模型强化学习是目前强化学习研究的主流,但折扣因子的选取使得近期期望报酬的影响大于远期期望报酬的影响,而有时候较大远期期望报酬的策略有可能是最优的,因此比较合理的方法是采用平均报酬模型强化学习。该文介绍了平均报酬模型强化学习的两个主要算法以及主要应用。     

结合LSTM的强化学习动态环境路径规划算法

在路径规划领域已经涌现出了诸多的优秀的经典算法,但这些传统方法往往基于静态环境,对于动态可变环境缺乏处理能力.本文提出一种结合LSTM强化学习动态环境路径规划算法.首先,本文以环境图像作为输入,最大限度了保证了原始的信息来源.而后构建了自动编码器用来对环境图像进行特征降维,降低了整体模型的复杂程度.最后采用深度强化学习...     

基于深度强化学习的移动机器人动态路径规划算法

为了在复杂舞台环境下使用移动机器人实现物品搬运或者载人演出,提出了一种基于深度强化学习的动态路径规划算法。首先通过构建全局地图获取移动机器人周围的障碍物信息,将演员和舞台道具分别分类成动态障碍物和静态障碍物。然后建立局部地图,通过LSTM网络编码动态障碍物信息,使用社会注意力机制计算每个动态障碍物的重要性来实现更好的避障效果。通过构建新的奖励函数来实现对动静态障碍物的不同躲避情况。最后通过模仿学习和优先级经验回放技术来提高网络的收敛速度,从而实现在舞台复杂环境下的移动机器人的动态路径规划。实验结果表明,该网络的收敛速度明显提高,在不同障碍物环境下都能够表现出好的动态避障效果。     

平均和折扣准则MDP基于TD(0)学习的统一NDP方法

为适应实际大规模M arkov系统的需要,讨论M arkov决策过程(MDP)基于仿真的学习优化问题.根据定义式,建立性能势在平均和折扣性能准则下统一的即时差分公式,并利用一个神经元网络来表示性能势的估计值,导出参数TD(0)学习公式和算法,进行逼近策略评估;然后,根据性能势的逼近值,通过逼近策略迭代来实现两种准则下统一的神经元动态规划(neuro-dynam ic programm ing,NDP)优化方法.研究结果适用于半M arkov决策过程,并通过一个数值例子,说明了文中的神经元策略迭代算法对两种准则都适用,验证了平均问题是折扣问题当折扣因子趋近于零时的极限情况.     

一种结合Tile Coding的平均奖赏强化学习算法

平均奖赏强化学习是强化学习中的一类重要的非折扣最优性框架,目前大多工作都主要是在离散域进行.本文尝试将平均奖赏强化学习算法和函数估计结合来解决连续状态空间的问题,并根据状态域的改变,相应修改R-learning和G-learning中参数的更新条件.此外对结合函数估计的G-learning算法的性能表现及其对各种参数的敏感程度进行针对性研究.最后给出实验结果及分析.实验结果证明R-learning和G-learning在ε较小的情况下解容易发散,同时也说明特征抽取方法Tile Coding的有效性,且可作为其它特征抽取方法的参考标准.     

强化学习系统及其基于可靠度最优的学习算法

归纳了强化学习的主要理论方法，提出了一个区分主客观因素的强化学习系统描述，引入了任务域的概念，针对以往强化学习采用的期望最优准则描述任务域能力的不足，考虑了目标水平准则下的首达时间可靠度优准则模型，分别结合随机逼近理论和时间差分理论，提出了基于概率估计的Ｊ－学习和无需建增量Ｒ－学习。     

基于Markov对策的多Agent强化学习模型及算法研究

在ＭＤＰ,单Ａｇｅｎｔ可以通过强化学习来寻找问题的最优解。但在多Ａｇｅｎｔ系统中,ＭＤＰ模型不再适用。同样极小极大Ｑ算法只能解决采用零和对策模型的ＭＡＳ学习问题。文中采用非零和Ｍａｒｋｏｖ对策作为多Ａｇｅｎｔ系统学习框架,并提出元对策强化学习的学习模型和元对策Ｑ算法。理论证明元对策Ｑ算法收敛在非零和Ｍａｒｋｏｖ对策的元对策最优解。     

Reinforcement Learning with Replacing Eligibility Traces

The eligibility trace is one of the basic mechanisms used in reinforcement learning to handle delayed reward. In this paper we introduce a new kind of eligibility trace, the replacing trace, analyze it theoretically, and show that it results in faster, more reliable learning than the conventional trace. Both kinds of trace assign credit to prior events according to how recently they occurred, but only the conventional trace gives greater credit to repeated events. Our analysis is for conventional and replace-trace versions of the offline TD(1) algorithm applied to undiscounted absorbing Markov chains. First, we show that these methods converge under repeated presentations of the training set to the same predictions as two well known Monte Carlo methods. We then analyze the relative efficiency of the two Monte Carlo methods. We show that the method corresponding to conventional TD is biased, whereas the method corresponding to replace-trace TD is unbiased. In addition, we show that the method corresponding to replacing traces is closely related to the maximum likelihood solution for these tasks, and that its mean squared error is always lower in the long run. Computational results confirm these analyses and show that they are applicable more generally. In particular, we show that replacing traces significantly improve performance and reduce parameter sensitivity on the "Mountain-Car" task, a full reinforcement-learning problem with a continuous state space, when using a feature-based function approximator.     

A CONSTRAINED ARCHITECTURE FOR LEARNING AND PROBLEM SOLVING

This paper describes Eureka , a problem-solving architecture that operates under strong constraints on its memory and processes. Most significantly, Eureka does not assume free access to its entire long-term memory. That is, failures in problem solving may arise not only from missing knowledge, but from the (possibly temporary) inability to retrieve appropriate existing knowledge from memory. Additionally, the architecture does not include systematic backtracking to recover from fruitless search paths. These constraints significantly impact Eureka 's design. Humans are also subject to such constraints, but are able to overcome them to solve problems effectively. In Eureka 's design, we have attempted to minimize the number of additional architectural commitments, while staying faithful to the memory constraints. Even under such minimal commitments, Eureka provides a qualitative account of the primary types of learning reported in the literature on human problem solving. Further commitments to the architecture would refine the details in the model, but the approach we have taken de-emphasizes highly detailed modeling to get at general root causes of the observed regularities. Making minimal additional commitments to Eureka 's design strengthens the case that many regularities in human learning and problem solving are entailments of the need to handle imperfect memory.     

多元轨迹同步化问题的改进型DTW算法

针对间歇生产过程故障诊断中各批次数据轨迹时间长度不一致,导致应用MPCA/MPLS 准确建模及故障诊断失效的问题,本文将动态时间错位（DTW）算法用于所观测的数据轨迹 时间长度的同步化;并在此基础上进一步提出了递推式和粗格子点两种DTW改进算法．仿真 结果表明,两种DTW改进算法能够有效地同步化各批次轨迹,而且可以推广到在线实时应用 ．     

可重入生产系统的递阶增强型学习调度

对平均报酬型马氏决策过程,本文研究了一种递阶增强型学习算法;并将算法应用 于一个两台机器组成的闭环可重入生产系统,计算机仿真结果表明,调度结果优于熟知的两 种启发式调度策略．     

Self-teaching adaptive dynamic programming for Gomoku

In this paper adaptive dynamic programming (ADP) is applied to learn to play Gomoku. The critic network is used to evaluate board situations. The basic idea is to penalize the last move taken by the loser and reward the last move selected by the winner at the end of a game. The results show that the presented program is able to improve its performance by playing against itself and has approached the candidate level of a commercial Gomoku program called 5-star Gomoku. We also examined the influence of two methods for generating games: self-teaching and learning through watching two experts playing against each other and presented the comparison results and reasons.     

遗传算法求解完全欺骗性问题的平均计算时间

讨论了一类遗传算法求解完全欺骗性问题的平均计算时间,证明了这类算法的平均计算时间是问题规模的指数次方,同时指出存在一种基于多重结构的遗传算法,它求解这个问题的２计算时间是问题规模的多项式次方。     

动态环境中基于增强式学习的路径规划方法

本文结合机器人路径规划问题介绍了增强式学习方法 ,实现了动态环境中基于增强式学习的自适应路径规划 .增强式学习通过采用随机性的控制策略 ,实现策略的优化搜索和在线学习 .并采用具有模式增强输入的BP网络进行决策参数估计 ,加快学习的收敛 .仿真试验证明该方法能有效实现动态环境中机器人的避碰和导航     

Approximate Policy Optimization and Adaptive Control in Regression Models

In this paper we use recent advances in approximate dynamic programming to develop an approximate policy optimization procedure that uses Monte Carlo simulations for numerical solution of dynamic optimization problems in economics. The procedure is applied to the classical problem of “learning by doing” in regression models, for which the value and extent of active experimentation are demonstrated in a variety of numerical studies.