Incremental online sparsification for model learning in real-time robot control

For many applications such as compliant, accurate robot tracking control, dynamics models learned from data can help to achieve both compliant control performance as well as high tracking quality. Online learning of these dynamics models allows the robot controller to adapt itself to changes in the dynamics (e.g., due to time-variant nonlinearities or unforeseen loads). However, online learning in real-time applications - as required in control - cannot be realized by straightforward usage of off-the-shelf machine learning methods such as Gaussian process regression or support vector regression. In this paper, we propose a framework for online, incremental sparsification with a fixed budget designed for fast real-time model learning. The proposed approach employs a sparsification method based on an independence measure. In combination with an incremental learning approach such as incremental Gaussian process regression, we obtain a model approximation method which is applicable in real-time online learning. It exhibits competitive learning accuracy when compared with standard regression techniques. Implementation on a real Barrett WAM robot demonstrates the applicability of the approach in real-time online model learning for real world systems.     

增量式人体姿态映射模型的学习方法

判别式3D人体姿态估计方法直接学习图像观测到姿态之间的映射,需要大量训练集,而GPR对这种大训练集的映射模型学习由于计算复杂度太高而受到极大限制。提出了一种基于GPR和LWPR的增量式映射模型的学习方法,利用GPR学习各局部映射模型,基于LWPR的思想在线调整现有的模型和训练新的局部模型以及姿态估计。实验表明,该方法能够极大地减少大数据集上高斯过程回归的计算代价,并获得准确的姿态估计。     

Statistical Learning for Humanoid Robots

The complexity of the kinematic and dynamic structure of humanoid robots make conventional analytical approaches to control increasingly unsuitable for such systems. Learning techniques offer a possible way to aid controller design if insufficient analytical knowledge is available, and learning approaches seem mandatory when humanoid systems are supposed to become completely autonomous. While recent research in neural networks and statistical learning has focused mostly on learning from finite data sets without stringent constraints on computational efficiency, learning for humanoid robots requires a different setting, characterized by the need for real-time learning performance from an essentially infinite stream of incrementally arriving data. This paper demonstrates how even high-dimensional learning problems of this kind can successfully be dealt with by techniques from nonparametric regression and locally weighted learning. As an example, we describe the application of one of the most advanced of such algorithms, Locally Weighted Projection Regression (LWPR), to the on-line learning of three problems in humanoid motor control: the learning of inverse dynamics models for model-based control, the learning of inverse kinematics of redundant manipulators, and the learning of oculomotor reflexes. All these examples demonstrate fast, i.e., within seconds or minutes, learning convergence with highly accurate final peformance. We conclude that real-time learning for complex motor system like humanoid robots is possible with appropriately tailored algorithms, such that increasingly autonomous robots with massive learning abilities should be achievable in the near future.     

Adaptive Inverse Control of Excitation System with Actuator Uncertainty

In this paper, we present a new approach of designing adaptive inverse controller for synchronous generator excitation system containing nonsmooth nonlinearities in actuator device. The proposed controller considers not only the dynamics of generator but also nonlinearities in actuator. To address such a challenge, support vector machines (SVM) is adopted to identify the plant and to construct the inverse controller. SVM networks, used to compensate nonlinearities in synchronous generator as well as in actuator, are adjusted online by an adaptive law via back propagation (BP) algorithm. To guarantee convergence and for fast learning, adaptive learning rate and convergence theorem are developed. Simulation results are given, showing satisfactory control performance and illustrate the potential of the proposed adaptive inverse controller as useful for practical purpose.     

Incremental online learning in high dimensions

Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of-possibly redundant-inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.     

Neural network-based adaptive controller design of roboticmanipulators with an observer

A neural network (NN)-based adaptive controller with an observer is proposed for the trajectory tracking of robotic manipulators with unknown dynamics nonlinearities. It is assumed that the robotic manipulator has only joint angle position measurements. A linear observer is used to estimate the robot joint angle velocity, while NNs are employed to further improve the control performance of the controlled system through approximating the modified robot dynamics function. The adaptive controller for robots with an observer can guarantee the uniform ultimate bounds of the tracking errors and the observer errors as well as the bounds of the NN weights. For performance comparisons, the conventional adaptive algorithm with an observer using linearity in parameters of the robot dynamics is also developed in the same control framework as the NN approach for online approximating unknown nonlinearities of the robot dynamics. Main theoretical results for designing such an observer-based adaptive controller with the NN approach using multilayer NNs with sigmoidal activation functions, as well as with the conventional adaptive approach using linearity in parameters of the robot dynamics are given. The performance comparisons between the NN approach and the conventional adaptation approach with an observer is carried out to show the advantages of the proposed control approaches through simulation studies     

Wavenet fuzzy PID controller for nonlinear MIMO systems: Experimental validation on a high-end haptic robotic interface

A novel global PID control scheme for nonlinear MIMO systems is proposed and implemented for a robot as study case, this scheme is called AWFPID from its adaptive wavelet fuzzy PID control structure. Basically, it identifies inverse error dynamics using a radial basis neural network with daughter RASP1 wavelets activation function; its output is in cascaded with an infinite impulse response (IIR) filter to prune irrelevant signals and nodes as well as to recover a canonical form. Then, online adaptive fuzzy tuning of a discrete PID regulator is proposed, whose closed-loop guarantees global regulation for nonlinear dynamical plants. The wavelet network includes a fuzzy inference system for online tuning of learning rates. A real-time experimental study on a three degrees of freedom haptic interface, the PHANToM Premium 1.0A, highlights the regulation with smooth control effort without using the mathematical model of the robot.     

A recursive modified partial least square aided data-driven predictive control with application to continuous stirred tank heater

In this paper, a data-driven predictive control strategy for nonlinear system is proposed and testified on a continuous stirred tank heater (CSTH) benchmark. A recursive modified partial least square (RMPLS) algorithm is employed to regress the local linear model. The algorithm of locally weighted projection regression (LWPR) is then leveraged to build the predictive model, based on which a novel data-driven predictive control strategy is put forward. The proposed predictive controller has the ability to deal with changing working conditions, benefiting from the incremental learning ability of RMPLS and LWPR. The performance of the proposed control strategy is demonstrated with the CSTH while the superiority is illustrated by comparison with an existing model-free adaptive control approach.     

Robot learning control based on recurrent neural network inverse model

A non-linear model-based feedforward, feedback, and learning controller is presented. This controller can control a non-linear plant such as a robot whose dynamics are initially unknown. In the feedforward part, a recurrent neural network (RNN) is used to model the inverse dynamics of the plant. In the feedback part, a PD controller is added to handle unmodeled dynamics and disturbances. Furthermore, an add-on learning controller is established to reduce tracking errors for repetitive tasks. The controller is validated with the control of a simulated two-joint manipulator. Simulation results show that the controller can successfully learn the inverse dynamics of a robot, perform accurate tracking for a general trajectory, and improve its own performance over the repetitions of a trajectory, with and without a payload change. © 1997 John Wiley & Sons, Inc.     

乒乓球机器人的视觉伺服系统

视觉伺服的乒乓球机器人系统作为典型的"手眼系统",是研究高速视觉感知和快速伺服运动的理想平台,其涉及的高速物体识别跟踪、快速精确轨迹预测及机械臂伺服准确回球等关键技术在工业、军事等领域有广泛的应用前景.本文提出了乒乓球机器人的高速视觉伺服系统实现方法,包括基于特征直方图统计和快速轮廓搜索的目标识别算法,基于模型参数学习和自适应模型调整的物体运动状态估计和轨迹预测算法,及基于轨迹预测的灵巧臂回球规划算法.通过实验验证了各算法的实时性和高效性,并在165cm高的仿人机器人"悟"和"空"上成功实现了双机器人对打和与人对打任务.     

Neutral learning of constrained nonlinear transformations

Two issues that are fundamental to developing autonomous intelligent robots, namely rudimentary learning capability and dexterous manipulation, are examined. A powerful neural learning formalism is introduced for addressing a large class of nonlinear mapping problems, including redundant manipulator inverse kinematics, commonly encountered during the design of real-time adaptive control mechanisms. Artificial neural networks with terminal attractor dynamics are used. The rapid network convergence resulting from the infinite local stability of these attractors allows the development of fast neural learning algorithms. Approaches to manipulator inverse kinematics are reviewed, the neurodynamics model is discussed, and the neural learning algorithm is presented     

Fast implementation of robot inverse dynamics with distributedarithmetic via a SIMD architecture

Real-time sophisticated robot control schemes require the online evaluation of the inverse robot dynamics model, a computationally intensive task. Research efforts to create efficient implementations of inverse models belong to the domain of computational robot dynamics. The current trends in this domain postulate that major improvements in computational efficiency can be achieved only in the context of customizing the dynamics formulations for specific robots and organizing the numerical computation. In this context, the objective of this paper is to introduce a novel SIMD parallel architecture based upon the distributed arithmetic technique for the efficient implementation of the general-purpose inverse robot dynamic problem. The approach is embedded in the Lagrangian formulation of robot dynamics with all equations expanded symbolically in their scalar form. In the proposed architecture any inherent parallelism of the computations is exploited to partition the inverse dynamics problem into discrete subtasks. The efficiency of the procedure is illustrated via the positioning system of the Puma robot     

SYMBOLIC COMPUTATION OF ROBOT DYNAMIC PARAMETERS

Optimal performance of robot manipulators can be achieved only by utilizing advanced control algorithms. However, precise control of robot motion requires the use of accurate dynamic models, which are very complicated due to varying arm geometric configuration, uncertain effects of load handling on the dynamic stability of the arm, and the high degree of nonlinearty and coupling exhibited between different links. Therefore, an efficient and fast method for on-line tuning of robot dynamic parameters must be devised. In this work a simplified model based on Lagrange-Euler dynamics is developed. The proposed method is simple and systematic for the extraction and identification of robot dynamic parameters. The dynamic parameters are then formulated as a regression model. This model is used to generate the closed-form solution of the dynamics. The analysis in this work is based on a set of compiled data for the Stanford arm to facilitate the study of the dynamic performance and closed-loop solutions of robot manipulators. For the derivation of the dynamics MAPLE (symbolic computer algebra language) is used.     

Indirect robot model learning for tracking control

Learning task-space tracking control on redundant robot manipulators is an important but difficult problem. A main difficulty is the non-uniqueness of the solution: a task-space trajectory has multiple joint-space trajectories associated, therefore averaging over non-convex solution space needs to be done if treated as a regression problem. A second class of difficulties arise for those robots when the physical model is either too complex or even not available. In this situation machine learning methods may be a suitable alternative to classical approaches. We propose a learning framework for tracking control that is applicable for underactuated or non-rigid robots where an analytical physical model of the robot is unavailable. The proposed framework builds on the insight that tracking problems are well defined in the joint task- and joint-space coordinates and consequently predictions can be obtained via local optimization. Physical experiments show that state-of-the art accuracy can be achieved in both online and offline tracking control learning. Furthermore, we show that the presented method is capable of controlling underactuated robot architectures as well.     

Visual motor control of a 7 DOF robot manipulator using a fuzzy SOM network

A fuzzy self-organizing map (SOM) network is proposed in this paper for visual motor control of a 7 degrees of freedom (DOF) robot manipulator. The inverse kinematic map from the image plane to joint angle space of a redundant manipulator is highly nonlinear and ill-posed in the sense that a typical end-effector position is associated with several joint angle vectors. In the proposed approach, the robot workspace in image plane is discretized into a number of fuzzy regions whose center locations and fuzzy membership values are determined using a Fuzzy C-Mean (FCM) clustering algorithm. SOM network then learns the inverse kinematics by on-line by associating a local linear map for each cluster. A novel learning algorithm has been proposed to make the robot manipulator to reach a target position. Any arbitrary level of accuracy can be achieved with a number of fine movements of the manipulator tip. These fine movements depend on the error between the target position and the current manipulator position. In particular, the fuzzy model is found to be better as compared to Kohonen self-organizing map (KSOM) based learning scheme proposed for visual motor control. Like existing KSOM learning schemes, the proposed scheme leads to a unique inverse kinematic solution even for a redundant manipulator. The proposed algorithms have been successfully implemented in real-time on a 7 DOF PowerCube robot manipulator, and results are found to concur with the theoretical findings.     

Hybrid trigonometric compound function neural networks for tracking control of a nonholonomic mobile robot

The purpose of this paper is to propose a hybrid trigonometric compound function neural network (NN) to improve the NN-based tracking control performance of a nonholonomic mobile robot with nonlinear disturbances. In the mobile robot control system, two NN controllers embedded in the closed-loop control system have the simple continuous learning and rapid convergence capability without the dynamics information of the mobile robot to realize the tracking control of the mobile robot. The neuron functions of the hidden layer in the three-layer feedforward network structure consist of the compound cosine function and the compound sine function combining a cosine or a sine function with a unipolar sigmoid function. The main advantages of this NN-based mobile robot control system are better real-time control capability and control accuracy by use of the proposed NN controllers for a nonholonomic mobile robot with nonlinear disturbances. Through simulation experiments applied to the nonholonomic mobile robot with the nonlinear disturbances of dynamics uncertainty and external disturbances, the simulation results show that the proposed NN control system of a nonholonomic mobile robot has better real-time control capability and control accuracy than the compound cosine function NN control system of a nonholonomic mobile robot and then verify the effectiveness of the proposed hybrid trigonometric compound function NN controller for improving the tracking control performance of a nonholonomic mobile robot with nonlinear disturbances.     

Estimation of physical interaction between a musculoskeletal robot and its surroundings

Recently, robots are expected to support our daily lives in real environments. In such environments, however, there are a lot of obstacles and the motion of the robot is affected by them. In this research, we develop a musculoskeletal robotic arm and a system identification method for coping with external forces while learning the dynamics of complicated situations, based on Gaussian process regression (GPR). The musculoskeletal robot has the ability to cope with external forces by utilizing a bio-inspired mechanism. GPR is an easy-to-implement method, but can handle complicated prediction tasks. The experimental results show that the behavior of the robot while interacting with its surroundings can be predicted by our method.     

Parameterizing robust manipulator controllers under approximate inverse dynamics: A double‐Youla approach

We consider the goal of ensuring robust stability when a given manipulator feedback control law is modified online, for example, to safely improve the performance by a learning module. To this end, the factorization approach is applied to both the plant and controller models to characterize robustly stabilizing controllers for rigid‐body manipulators under approximate inverse dynamics control. Outer‐loop controllers to stabilize the nonlinear uncertain loop that results from approximate inverse dynamics are often derived by lumping uncertainty in a single term and subsequent analysis of the error system. Here, by contrast, the well‐known norm bounds of these uncertain dynamics are first recast into a generalized plant configuration that preserves the characteristic uncertainty structure. Then, the overall loop uncertainty is expressed with respect to the nominal outer‐loop feedback controller by means of an uncertain dual‐Youla operator. Therefore, using the dual‐Youla parameterization, we provide a novel way to rigorously quantify permissible perturbations of robot manipulator feedforward/feedback controllers. The method proposed in this paper does not constitute another robust control law for rigid‐body manipulators, but rather a characterization of a set of robustly stabilizing controllers. The resulting double‐Youla parameterization for the control of robot manipulators is amenable to numerous advanced design methods. The result is thoroughly discussed by a planar elbow manipulator and exemplified with a six‐degree‐of‐freedom robot scenario with varying payload.     

Online reduced gaussian process regression based generalized likelihood ratio test for fault detection

In this paper we consider a new fault detection approach that merges the benefits of Gaussian process regression (GPR) with a generalized likelihood ratio test (GLRT). The GPR is one of the most well-known machine learning techniques. It is simpler and generally more robust than other methods. To deal with both high computational costs for large data sets and time-varying dynamics of industrial processes, we consider a reduced and online version of the GPR method. The online reduced GPR (ORGPR) aims to select a reduced set of kernel functions to build the GPR model and apply it for online fault detection based on GLRT chart. Compared with the conventional GPR technique, the proposed ORGPR method has the advantages of improving the computational efficiency by decreasing the dimension of the kernel matrix. The developed ORGPR-based GLRT (ORGPR-based GLRT) could improve the fault detection efficiency since it is able to track the time-varying characteristics of the processes. The fault detection performance of the developed ORGPR-based GLRT method is evaluated using a Tennessee Eastman process. The simulation results show that the proposed method outperforms the conventional GPR-based GLRT technique.     

A learning method for multivariable PID control synthesis based on estimated plant Jacobian

Despite decades of research involving optimal control of multivariable systems, such controllers require accurate linear models of the plant dynamics. Real systems contain nonlinearities and high-order dynamics that may be difficult to model using conventional techniques. This paper presents a novel learning control (LC) method for PID controllers that doesn’t require explicit modeling of the plant dynamics. This method utilizes gradient descent techniques to iteratively reduce an error-related objective function. Simulations involving a hydrofoil catamaran show that the proposed PID-LC algorithm improves controller performance compared to LQR controllers derived from multivariable models.