基于多演示动作基元参数化学习的机器人任务泛化

针对机器人示范学习过程中任务泛化与动作轨迹泛化问题,提出了一种将多演示动作轨迹的任务参数化学习与动作序列推理相结合的方法.针对通用动作基元的多演示轨迹样本,利用动态运动基元进行轨迹编码并建立任务参数化模型,利用高斯过程回归学习外部参数与模型参数之间的映射.针对新的任务实例,利用规划域定义语言推理缺失动作序列,任务参数化模型根据新的外部参数泛化出动作的目标轨迹,并修正轨迹误差.在UR5机器人上的实验表明,面对不同任务实例和环境变化,该方法可灵活生成动作序列并调整泛化目标,基于多演示的任务参数化模型能够对给定外部参数泛化出平滑的目标轨迹,泛化效果优于单一演示轨迹,提高了机器人任务泛化的能力.     

机器人运动轨迹的模仿学习综述EI北大核心CSCD

作为机器人技能学习中的一个重要分支,模仿学习近年来在机器人系统中得到了广泛的应用.模仿学习能够将人类的技能以一种相对直接的方式迁移到机器人系统中,其思路是先从少量示教样本中提取相应的运动特征,然后将该特征泛化到新的情形.本文针对机器人运动轨迹的模仿学习进行综述.首先详细解释模仿学习中的技能泛化、收敛性和外插等基本问题;其次从原理上对动态运动基元、概率运动基元和核化运动基元等主要的模仿学习算法进行介绍;然后深入地讨论模仿学习中姿态和刚度矩阵的学习问题、协同和不确定性预测的问题以及人机交互中的模仿学习等若干关键问题;最后本文探讨了结合因果推理的模仿学习等几个未来的发展方向.     

基于STFT的仿人机器人手臂轨迹生成方法

为了提高机器人轨迹生成算法的泛化性,提出了一种基于时间-空间特征模板(STFT,spatiotemporal feature template)的机器人手臂轨迹生成方法.首先,针对机器人示教轨迹往往存在的时间长短和幅度差异较大的问题,采用广义的典型时间规整(generalized canonical time warping,GCTW)方法来统一时间和幅度的变化,从而获取机器人示教轨迹的共同特征模板.其次,基于STFT,引入机器人轨迹生成的平滑性约束、任务约束等因素,设定轨迹生成的目标函数并优化.最终在NAO仿人机器人平台上验证了所提出的轨迹生成算法,基于STFT生成机器人弧形轨迹并完成数字书写.实验结果表明,本文提出的基于STFT的轨迹生成策略可以生成满足期望条件的机器人轨迹,并具有一定的泛化性.     

基于能效优化的仿人机器人跑步步态优化与控制

针对高能耗导致的仿人机器人难以大规模实用化的问题,提出了一种新的仿人机器人参数化跑步步态优化方法。分析了不同跑步步态参数对仿人机器人水平、垂直方向的稳定性及能耗的影响,将机器人步态优化问题转化为对步态参数的多目标寻优问题,根据连杆模型得到机器人跑步过程中水平、垂直方向的稳定裕度及能耗表达式,并构造目标函数,采用基于对位学习的遗传算法对机器人参数化跑步步态进行多目标寻优,在保证机器人俯仰、翻滚和偏摆各方向力矩平衡的前提下降低整体能量消耗;针对传统遗传算法早熟及收敛速度慢的问题,提出基于领域知识的精细化初始成员策略,采取生成种群成员对位点的方式更新种群,以加快收敛速度;为提高轨迹跟踪性能,设计了自适应控制器,并给出了稳定性证明。仿真实验表明:该方法能有效降低能耗并保证其稳定性。     

基于路径积分强化学习方法的蛇形机器人目标导向运动

路径积分方法源于随机最优控制,是一种数值迭代方法,可求解连续非线性系统的最优控制问题,不依赖于系统模型,快速收敛.文中将基于路径积分强化学习的策略改善方法用于蛇形机器人的目标导向运动.使用路径积分强化学习方法学习蛇形机器人步态方程的参数,不仅可以在仿真环境下使蛇形机器人规避障碍到达目标点,利用仿真环境的先验知识也能在实际环境下快速完成相同的任务.实验结果验证方法的正确性.     

机器人模仿学习的非接触观测控制图模型

提出一种新的基于非接触观测信息的机器人模仿学习表征与执行的控制图模型.建立可模仿学习的人-机关系,并得出模仿学习前提条件是以系统末端微分运动为基本行为元.提出控制图模型结构和基于视觉观测序列的模型学习方法.提出基于累积和瞬时相关函数的观测序列分割和图结构生成方法,和基于RBF(径向基函数)网络的行为元目标学习方法.通过不同结构和自由度的机器人毛笔绘画和物体抓取模仿学习实例实验,证明了所提出模型在视觉观测信息下能够表征与执行不同层次和类型的行为,具有良好的泛化能力、通用性及实用性.     

双足机器人动态步行实时步态生成

针对双足机器人动态步行生成关节运动轨迹复杂问题,提出了一种简单直观的实时步态生成方案。建立了平面五杆双足机器人动力学模型,通过模仿人类步行主要运动特征并根据双足机器人动态步行双腿姿态变化的要求,将动态步行复杂任务分解为顺序执行的四个过程,在关节空间相对坐标系下设计了躯干运动模式、摆动腿和支撑腿动作及步行速度调整模式,结合当前步行控制结果反馈实时产生稳定的关节运动轨迹。仿真实验验证了该方法的有效性,简单易实现。     

迭代学习神经网络控制在机器人示教学习中的应用

示教学习是机器人运动技能获取的一种高效手段.当采用摄像机作为示教轨迹记录部件时,示教学习涉及如何通过反复尝试获得未知机器人摄像机模型问题.本文力图针对非线性系统重复作业中的可重复不确定性学习,提出一个迭代学习神经网络控制方案,该控制器将保证系统最大跟踪误差维持在神经网络有效近似域内.为此提出了一个适合于重复作业应用的分布式神经网络结构.该神经网络由沿期望轨线分布的一系列局部神经网络构成,每一局部神经网络对对应期望轨迹点邻域进行近似并通过重复作业完成网络训练.由于所设计的局部神经网络相互独立,因此一个全程轨迹可以通过分段训练完成,由起始段到结束段,逐段实现期望轨迹的准确跟踪.该方法在具有未知机器人摄像机模型的轨迹示教模仿中得到验证,显示了它是一种高效的训练方法,同时具有一致的误差限界能力.     

极坐标下基于迭代学习的移动机器人轨迹跟踪控制

为提高自主移动机器人对一类特殊轨迹的重复跟踪能力,在极坐标下建立了3轮全向移动机器人的运动学模型,结合离散时域下对轨迹跟踪问题的描述方法,采用开闭环P型迭代学习控制算法,并在给定条件下证明了其收敛性,随着迭代次数的增加,该算法能够有效改善动态不确定环境中系统的稳定性与收敛的快速性。通过将仿真结果作用于实际动态系统的初始控制输入,从而在实际环境下能以较少的迭代过程来获取控制律。实验结果表明,在仿真环境下机器人可以较好地跟踪玫瑰曲线,在实际机器人测试中,机器人能够较好地跟踪期望轨迹,从而证实了该方法对提高自主移动机器人轨迹跟踪能力的可行性与有效性。     

基于动态运动基元的轨迹学习方法

针对动态运动基元轨迹学习方法得到的学习轨迹终点值存在较大位置误差的问题,提出一种通过增大动态运动基元积分步数来减小位置误差的方法.通过以正弦轨迹、斜坡轨迹为示教轨迹的仿真实验验证了该方法的有效性.针对动态运动基元学习轨迹起始值与目标值相同时得到的学习轨迹恒为直线的问题,提出一种分段式轨迹学习方法.以轨迹极值点为分界点将学习轨迹分割为多段初始值与目标值不同的轨迹,通过仿真实验验证了该方法的有效性.     

Imitating others by composition of primitive actions: A neuro-dynamic model

This paper introduces a novel neuro-dynamical model that accounts for possible mechanisms of action imitation and learning. It is considered that imitation learning requires at least two classes of generalization. One is generalization over sensory–motor trajectory variances, and the other class is on cognitive level which concerns on more qualitative understanding of compositional actions by own and others which do not necessarily depend on exact trajectories. This paper describes a possible model dealing with these classes of generalization by focusing on the problem of action compositionality. The model was evaluated in the experiments using a small humanoid robot. The robot was trained with a set of different actions concerning object manipulations which can be decomposed into sequences of action primitives. Then the robot was asked to imitate a novel compositional action demonstrated by a human subject which are composed from prior-learned action primitives. The results showed that the novel action can be successfully imitated by decomposing and composing it with the primitives by means of organizing unified intentional representation hosted by mirror neurons even though the trajectory-level appearance is different between the ones of observed and those of self-generated.     

Experience-based imitation using RNNPB

Robot imitation is a useful and promising alternative to robot programming. Robot imitation involves two crucial issues. The first is how a robot can imitate a human whose physical structure and properties differ greatly from its own. The second is how the robot can generate various motions from finite programmable patterns (generalization). This paper describes a novel approach to robot imitation based on its own physical experiences. We considered the target task of moving an object on a table. For imitation, we focused on an active sensing process in which the robot acquires the relation between the object's motion and its own arm motion. For generalization, we applied the RNNPB (recurrent neural network with parametric bias) model to enable recognition/generation of imitation motions. The robot associates the arm motion which reproduces the observed object's motion presented by a human operator. Experimental results proved the generalization capability of our method, which enables the robot to imitate not only motion it has experienced, but also unknown motion through nonlinear combination of the experienced motions.     

仿人机器人步态平衡泛化模型的建立与仿真

通过人体示教计算零力矩点(zero moment point, ZMP),并通过补偿关节角度对其矫正的方法可以解决机器人步行不稳定的问题,但仍存在算法复杂度过高等问题。本文提出一种人体示教与机器学习相结合的方法,基于支持向量回归算法建立机器人的步态平衡泛化模型,通过该模型可以实现对模型输入人体示教的关节角度和ZMP信息后直接得到经稳定性补偿的关节角度,并以此驱动机器人完成步行动作。引入鲸鱼优化算法(whale optimization algorithm, WOA)优化模型的参数以使模型得到最优的泛化效果,完善步态平衡模型的性能。WEBOTS仿真平台下,使用模型输出的补偿后的关节角度驱动NAO机器人,其动作自然、稳定且算法复杂度较低,验证了本文方法的可行性。     

Learning neural-shaped quadratic Lyapunov function for stable,accurate and generalizable human–robot skills transfer

Learning a stable dynamic system (DS) encoding human motion rules has been shown as an efficient approach for transferring motion skills. However, contradictions always exist between the stability, accuracy and generalization of the learned DS. This paper presents an approach to enhance the accuracy and generalization by learning a neural-shaped quadratic Lyapunov function (NS-QLF). For the stability concern, the NS-QLF is designed to satisfy LF basic properties. Thanks to the flexibility of the neural network, the NS-QLF shape can capture motion rules in a broad area. The corresponding neural-learning problem is formulated as a convex optimization problem. We then learn an original DS (ODS) by using the Gaussian process regression (GPR) algorithm and stabilize the ODS by solving an NS-QLF-constrained convex optimization problem. The resulted stable DS (SDS) can not only accurately reproduce trajectories near the demonstration area, but also can utilize the NS-QLF shape information to enhance the generalization capacity in regions away from the demonstration area. Various comparative simulations and experiments are conducted to show the benefits of the presented approach.     

Indirect iterative learning control for a discrete visual servo without a camera-robot model.

This paper presents a discrete learning controller for vision-guided robot trajectory imitation with no prior knowledge of the camera-robot model. A teacher demonstrates a desired movement in front of a camera, and then, the robot is tasked to replay it by repetitive tracking. The imitation procedure is considered as a discrete tracking control problem in the image plane, with an unknown and time-varying image Jacobian matrix. Instead of updating the control signal directly, as is usually done in iterative learning control (ILC), a series of neural networks are used to approximate the unknown Jacobian matrix around every sample point in the demonstrated trajectory, and the time-varying weights of local neural networks are identified through repetitive tracking, i.e., indirect ILC. This makes repetitive segmented training possible, and a segmented training strategy is presented to retain the training trajectories solely within the effective region for neural network approximation. However, a singularity problem may occur if an unmodified neural-network-based Jacobian estimation is used to calculate the robot end-effector velocity. A new weight modification algorithm is proposed which ensures invertibility of the estimation, thus circumventing the problem. Stability is further discussed, and the relationship between the approximation capability of the neural network and the tracking accuracy is obtained. Simulations and experiments are carried out to illustrate the validity of the proposed controller for trajectory imitation of robot manipulators with unknown time-varying Jacobian matrices.     

基于语义的软件服务故障自动诊断模型

模仿学习是机器人仿生机制研究的主要内容之一,即通过观察、理解、学习、模仿示教行为实现机器人的仿生特性。基于高斯过程分别表达采集离散示教信号所构成的示教轨迹和含有未知参数策略的模仿轨迹,构建模仿学习方法框架,将概率模型匹配引入到模仿学习中,以KL散度为代价函数比较两种轨迹的概率分布,运用梯度下降法寻求使KL散度最小的最优模仿控制策略,将策略应用于模仿机器人以完成与示教相同的模仿任务。以关节型机器人的机械臂摆动行为模仿为学习任务进行仿真,结果表明基于概率轨迹匹配的模仿学习方法能够实现机械臂摆动行为模仿,学习过程较传统方法简易且学习效果较好。     

模仿学习综述：传统与新进展

模仿学习是强化学习与监督学习的结合,目标是通过观察专家演示,学习专家策略,从而加速强化学习。通过引入任务相关的额外信息,模仿学习相较于强化学习,可以更快地实现策略优化,为缓解低样本效率问题提供了解决方案。模仿学习已成为解决强化学习问题的一种流行框架,涌现出多种提高学习性能的算法和技术。通过与图形图像学的最新研究成果相结合,模仿学习已经在游戏人工智能（artificial intelligence,AI）、机器人控制和自动驾驶等领域发挥了重要作用。本文围绕模仿学习的年度发展,从行为克隆、逆强化学习、对抗式模仿学习、基于观察量的模仿学习和跨领域模仿学习等多个角度进行深入探讨,介绍了模仿学习在实际应用上的最新情况,比较了国内外研究现状,并展望了该领域未来的发展方向。旨在为研究人员和从业人员提供模仿学习的最新进展,从而为开展工作提供参考与便利。     

On learning, representing, and generalizing a task in a humanoid robot.

We present a programming-by-demonstration framework for generically extracting the relevant features of a given task and for addressing the problem of generalizing the acquired knowledge to different contexts. We validate the architecture through a series of experiments, in which a human demonstrator teaches a humanoid robot simple manipulatory tasks. A probability-based estimation of the relevance is suggested by first projecting the motion data onto a generic latent space using principal component analysis. The resulting signals are encoded using a mixture of Gaussian/Bernoulli distributions (Gaussian mixture model/Bernoulli mixture model). This provides a measure of the spatio-temporal correlations across the different modalities collected from the robot, which can be used to determine a metric of the imitation performance. The trajectories are then generalized using Gaussian mixture regression. Finally, we analytically compute the trajectory which optimizes the imitation metric and use this to generalize the skill to different contexts.     

Acquiring Mobile Robot Behaviors by Learning Trajectory Velocities

The development of robots that learn from experience is a relentless challenge confronting artificial intelligence today. This paper describes a robot learning method which enables a mobile robot to simultaneously acquire the ability to avoid objects, follow walls, seek goals and control its velocity as a result of interacting with the environment without human assistance. The robot acquires these behaviors by learning how fast it should move along predefined trajectories with respect to the current state of the input vector. This enables the robot to perform object avoidance, wall following and goal seeking behaviors by choosing to follow fast trajectories near: the forward direction, the closest object or the goal location respectively. Learning trajectory velocities can be done relatively quickly because the required knowledge can be obtained from the robot's interactions with the environment without incurring the credit assignment problem. We provide experimental results to verify our robot learning method by using a mobile robot to simultaneously acquire all three behaviors.