迭代学习神经网络控制在机器人示教学习中的应用

示教学习是机器人运动技能获取的一种高效手段.当采用摄像机作为示教轨迹记录部件时,示教学习涉及如何通过反复尝试获得未知机器人摄像机模型问题.本文力图针对非线性系统重复作业中的可重复不确定性学习,提出一个迭代学习神经网络控制方案,该控制器将保证系统最大跟踪误差维持在神经网络有效近似域内.为此提出了一个适合于重复作业应用的分布式神经网络结构.该神经网络由沿期望轨线分布的一系列局部神经网络构成,每一局部神经网络对对应期望轨迹点邻域进行近似并通过重复作业完成网络训练.由于所设计的局部神经网络相互独立,因此一个全程轨迹可以通过分段训练完成,由起始段到结束段,逐段实现期望轨迹的准确跟踪.该方法在具有未知机器人摄像机模型的轨迹示教模仿中得到验证,显示了它是一种高效的训练方法,同时具有一致的误差限界能力.     

Reinforcement learning for robot soccer

Batch reinforcement learning methods provide a powerful framework for learning efficiently and effectively in autonomous robots. The paper reviews some recent work of the authors aiming at the successful application of reinforcement learning in a challenging and complex domain. It discusses several variants of the general batch learning framework, particularly tailored to the use of multilayer perceptrons to approximate value functions over continuous state spaces. The batch learning framework is successfully used to learn crucial skills in our soccer-playing robots participating in the RoboCup competitions. This is demonstrated on three different case studies.

Integrated architecture for industrial robot programming and control

As robot control systems are traditionally closed, it is difficult to add supplementary intelligence. Accordingly, as based on a new notion of user views, a layered system architecture is proposed. Bearing in mind such industrial demands as computing efficiency and simple factory-floor operation, the control layers are parameterized by means of functional operators consisting of pieces of compiled code that can be passed as parameters between the layers. The required interplay between application-specific programs and built-in motion control is thereby efficiently accomplished. The results from experimental evaluation and several case studies suggest the architecture to be very useful also in an industrial context.     

Bionic autonomous learning control of a two-wheeled self-balancing flexible robot

This paper presents an OCPA (operant conditioning probabilistic automaton) bionic autonomous learning system based on Skinner’s operant conditioning theory for solving the balance control problem of a two-wheeled flexible robot. The OCPA learning system consists of two stages: in the first stage, an operant action is selected stochastically from a set of operant actions and then used as the input of the control system; in the second stage, the learning system gathers the orientation information of the system and uses it for optimization until achieves control target. At the same time, the size of the operant action set can be automatically reduced during the learning process for avoiding little probability event. Theory analysis is made for the designed OCPA learning system in the paper, which theoretically proves the convergence of operant conditioning learning mechanism in OCPA learning system, namely the operant action entropy will converge to minimum with the learning process. And then OCPA learning system is applied to posture balanced control of two-wheeled flexible self-balanced robots. Robot does not have posutre balanced skill in initial state and the selecting probability of each operant in operant sets is equal. With the learning proceeding, the selected probabilities of optimal operant gradually tend to one and the operant action entropy gradually tends to minimum, and so robot gradually learned the posture balanced skill.     

存在环境约束的机器人自适应迭代学习控制

针对存在不确定性和外界干扰的受限机器人系统提出一种自适应迭代学习控制律.不确定性参数被估计在时间域内,同时重复性外界干扰在迭代域内得到补偿.通过引入饱和学习函数,保证了闭环系统所有信号有界.借助Lyapunov复合能量函数法,证明了系统渐进收敛到期望轨迹的同时,能够保证力跟踪误差有界可调.     

Adaptive iterative learning control for robot manipulators

In this paper, we propose some adaptive iterative learning control (ILC) schemes for trajectory tracking of rigid robot manipulators, with unknown parameters, performing repetitive tasks. The proposed control schemes are based upon the use of a proportional-derivative (PD) feedback structure, for which an iterative term is added to cope with the unknown parameters and disturbances. The control design is very simple in the sense that the only requirement on the PD and learning gains is the positive definiteness condition and the bounds of the robot parameters are not needed. In contrast to classical ILC schemes where the number of iterative variables is generally equal to the number of control inputs, the second controller proposed in this paper uses just two iterative variables, which is an interesting fact from a practical point of view since it contributes considerably to memory space saving in real-time implementations. We also show that it is possible to use a single iterative variable in the control scheme if some bounds of the system parameters are known. Furthermore, the resetting condition is relaxed to a certain extent for a certain class of reference trajectories. Finally, simulation results are provided to illustrate the effectiveness of the proposed controllers.     

基于大脑情感学习的四轮驱动机器人速度补偿控制

由于4轮驱动机器人的轮间耦合特性及系统非线性的存在,即使单个驱动电机的控制精度达到最优,机器人整体的运动控制效果也未必理想.针对这一问题,提出一种基于大脑情感学习的机器人速度补偿控制方法.基于大脑情感学习计算模型,设计了融合机器人整体速度跟踪误差及其积分、微分信息的补偿控制器,通过计算模型内部各节点权值的在线学习,及时地调整控制器的参数,实现对4个轮子速度的自适应补偿.仿真实验表明,该方法有效减小了非线性干扰对系统的影响,具有较高的稳态控制精度和较快的响应速度,大大提高了机器人整体的速度和轨迹跟踪精度.     

中值流辅助在线多示例目标跟踪

针对机器人演示学习中目标跟踪性能的严格要求,提出一种可以有效克服快速运动、遮挡和目标漂移的物体跟踪方法.首先计算中值流,并预测目标的位置偏移,以此计算高斯权重;然后修正搜索区域,并使用在线多示例分类器进行目标搜索,计算似然度;最后使用贝叶斯框架对结果进行融合,使用穷举搜索得到最优的预测位置,并更新在线分类器.实验结果表明,与现有方法相比,该方法对快速运动和目标漂移具有更强的鲁棒性,而且可以达到实时跟踪.     

A morphable template framework for robot learning by demonstration: Integrating one-shot and incremental learning approaches

Robot learning by demonstration is key to bringing robots into daily social environments to interact with and learn from human and other agents. However, teaching a robot to acquire new knowledge is a tedious and repetitive process and often restrictive to a specific setup of the environment. We propose a template-based learning framework for robot learning by demonstration to address both generalisation and adaptability. This novel framework is based upon a one-shot learning model integrated with spectral clustering and an online learning model to learn and adapt actions in similar scenarios. A set of statistical experiments is used to benchmark the framework components and shows that this approach requires no extensive training for generalisation and can adapt to environmental changes flexibly. Two real-world applications of an iCub humanoid robot playing the tic-tac-toe game and soldering a circuit board are used to demonstrate the relative merits of the framework.     

Nonlinear control with acceleration feedback for a two-link flexible robot

In this paper, a three-loop control strategy is applied to each link of a two-link flexible robot. In the first loop feedback linearization is applied to the rigid and motor dynamics. The second loop consists of a simple proportional-derivative (PD) control law for accurate rigid body angle tracking. The third loop uses endpoint accelearation feedback to account for flexure effects. The overall scheme is relatively simple in order to facilitate easy implementation; experimental results are provided to verify the effectiveness of the developed schemes.     

Velocity observer-based iterative learning control for robot manipulators

This article addresses the problem of designing an iterative learning control for trajectory tracking of rigid robot manipulators subject to external disturbances, and performing repetitive tasks, without using the velocity measurement. For solving this problem, a velocity observer having an iterative form is proposed to reconstruct the velocity signal in the control laws. Under assumptions that the disturbances are repetitive and the velocities are bounded, it has been shown that the whole control system (robot plus controller plus observer) is asymptotically stable and the observation error is globally asymptotically stable, over the whole finite time-interval when the iteration number tends to infinity. This proof is based upon the use of a Lyapunov-like positive definite sequence, which is shown to be monotonically decreasing under the proposed observer–controller schemes.     

Iterative learning control for integrated system of robot and machine tool

This study is concerned with the integrated system of a robot and a machine tool. The major task of robot is loading the workpiece to the machine tool for contour cutting. An iterative learning control (ILC) algorithm is proposed to improve the accuracy of the finished product. The proposed ILC is to modify the input command of the next machining cycle for both robot and machine tool to iteratively enhance the output accuracy of the robot and machine tool. The modified command is computed based on the current tracking/contour error. For the ILC of the robot, tracking error is considered as the control objective to reduce the tracking error of motion path, in particular, the error at the endpoint. Meanwhile, for the ILC of the machine tool, contour error is considered as the control objective to improve the contouring accuracy, which determines the quality of machining. In view of the complicated contour error model, the equivalent contour error instead of the actual contour error is taken as the control objective in this study. One challenge for the integrated system is that there exists an initial state error for the machine tool dynamics, violating the basic assumption of ILC. It will be shown in this study that the effects of initial state error can be significantly reduced by the ILC of the robot. The proposed ILC algorithm is verified experimentally on an integrated system of commercial robot and machine tool. The experimental results show that the proposed ILC can achieve more than 90% of reduction on both the RMS tracking error of the robot and the RMS contour error of the machine tool within six learning iterations. The results clearly validate the effectiveness of the proposed ILC for the integrated system.     

Learning from demonstration in robots: Experimental comparison of neural architectures

Robots have played an important role in the automation of computer aided manufacturing. The classical robot control implementation involves an expensive key step of model-based programming. An intuitive way to reduce this expensive exercise is to replace programming with machine learning of robot actions from demonstration where a (learner) robot learns an action by observing a demonstrator robot performing the same. To achieve this learning from demonstration (LFD) different machine learning techniques such as Artificial Neural Networks (ANN), Genetic Algorithms, Hidden Markov Models, Support Vector Machines, etc. can be used. This piece of work focuses exclusively on ANNs. Since ANNs have many standard architectural variations divided into two basic computational categories namely the recurrent networks and feed-forward networks, representative networks from each have been selected for study, i.e. Feed Forward Multilayer Perceptron (FF) network for feed-forward networks category and Elman (EL), and Nonlinear Autoregressive Exogenous Model (NARX) networks for the recurrent networks category. The main objective of this work is to identify the most suitable neural architecture for application of LFD in learning different robot actions. The sensor and actuator streams of demonstrated action are used as training data for ANN learning. Consequently, the learning capability is measured by comparing the error between demonstrator and corresponding learner streams. To achieve fairness in comparison three steps have been taken. First, Dynamic Time Warping is used to measure the error between demonstrator and learner streams, which gives resilience against translation in time. Second, comparison statistics are drawn between the best, instead of weight-equal, configurations of competing architectures so that learning capability of any architecture is not forced handicap. Third, each configuration's error is calculated as the average of ten trials of all possible learning sequences with random weight initialization so that the error value is independent of a particular sequence of learning or a particular set of initial weights. Six experiments are conducted to get a performance pattern of each architecture. In each experiment, a total of nine different robot actions were tested. Error statistics thus obtained have shown that NARX architecture is most suitable for this learning problem whereas Elman architecture has shown the worst suitability. Interestingly the computationally lesser MLP gives much lower and slightly higher error statistics compared to the computationally superior Elman and NARX neural architectures, respectively.     

A library for constraint consistent learning

This paper introduces the first, open source software library for Constraint Consistent Learning (CCL). It implements a family of data-driven methods that are capable of (i) learning state-independent and -dependent constraints, (ii) decomposing the behaviour of redundant systems into task- and null-space parts, and (iii) uncovering the underlying null space control policy. It is a tool to analyse and decompose many everyday tasks, such as wiping, reaching and drawing. The library also includes several tutorials that demonstrate its use with both simulated and real world data in a systematic way. This paper documents the methods contained within the library, including the implementations of said methods in tutorials and associated helper methods. The software is made freely available to the community, to enable code reuse and allow users to gain in-depth experience in statistical learning in this area.     

On the learning control of a robot manipulator

This paper derives a learning control law to achieve trajectory following for a robot manipulator. The controller consists of two parts, a computed torque servo for the rigid body terms that can be modelled and a learning law for the unmodelled dynamics. An advantage of this method is that bounds can be assigned to the position and velocity tracking errors.     

不确定机器人的自适应神经网络控制与学习

针对具有未知动态的电驱动机器人,研究其自适应神经网络控制与学习问题.首先,设计了稳定的自适应神经网络控制器,径向基函数(RBF)神经网络被用来逼近电驱动机器人的未知闭环系统动态,并根据李雅普诺夫稳定性理论推导了神经网络权值更新律.在对回归轨迹实现跟踪控制的过程中,闭环系统内部信号的部分持续激励(PE)条件得到满足.随着PE条件的满足,设计的自适应神经网络控制器被证明在稳定的跟踪控制过程中实现了电驱动机器人未知闭环系统动态的准确逼近.接着,使用学过的知识设计了新颖的学习控制器,实现了闭环系统稳定、改进了控制性能.最后,通过数字仿真验证了所提控制方法的正确性和有效性.     

基于迭代学习控制的移动机器人轨迹跟踪控制

为提高移动机器人对特定轨迹的重复跟踪能力,提出了采用开闭环PD型迭代学习控制算法对移动机器人进行轨迹跟踪控制的方法。建立了包含外界干扰的非完整约束条件下的轮式移动机器人运动学模型,给出了系统的控制算法和控制结构。仿真结果表明,采用开闭环PD型迭代学习控制算法对轨迹跟踪是可行有效的,收敛速度优于其他迭代学习算法。     

Indirect robot model learning for tracking control

Learning task-space tracking control on redundant robot manipulators is an important but difficult problem. A main difficulty is the non-uniqueness of the solution: a task-space trajectory has multiple joint-space trajectories associated, therefore averaging over non-convex solution space needs to be done if treated as a regression problem. A second class of difficulties arise for those robots when the physical model is either too complex or even not available. In this situation machine learning methods may be a suitable alternative to classical approaches. We propose a learning framework for tracking control that is applicable for underactuated or non-rigid robots where an analytical physical model of the robot is unavailable. The proposed framework builds on the insight that tracking problems are well defined in the joint task- and joint-space coordinates and consequently predictions can be obtained via local optimization. Physical experiments show that state-of-the art accuracy can be achieved in both online and offline tracking control learning. Furthermore, we show that the presented method is capable of controlling underactuated robot architectures as well.     

Incremental online sparsification for model learning in real-time robot control

For many applications such as compliant, accurate robot tracking control, dynamics models learned from data can help to achieve both compliant control performance as well as high tracking quality. Online learning of these dynamics models allows the robot controller to adapt itself to changes in the dynamics (e.g., due to time-variant nonlinearities or unforeseen loads). However, online learning in real-time applications - as required in control - cannot be realized by straightforward usage of off-the-shelf machine learning methods such as Gaussian process regression or support vector regression. In this paper, we propose a framework for online, incremental sparsification with a fixed budget designed for fast real-time model learning. The proposed approach employs a sparsification method based on an independence measure. In combination with an incremental learning approach such as incremental Gaussian process regression, we obtain a model approximation method which is applicable in real-time online learning. It exhibits competitive learning accuracy when compared with standard regression techniques. Implementation on a real Barrett WAM robot demonstrates the applicability of the approach in real-time online model learning for real world systems.     

基于边界层的不确定机器人自适应迭代学习控制

针对不确定的多连杆机械手的跟踪控制问题,提出一种基于边界层的自适应迭代学习控制方法.自适应控制用来估计系统的未知参数的上界,本文主要特征是基于边界层设计自适应迭代学习控制器,避免了传统方法设计控制器的不连续性,削弱抖振现象的同时也提高系统的鲁棒性.理论证明系统所有信号有界,系统误差渐进收敛到边界层邻域内.仿真表明了算法的有效性.