From dynamic movement primitives to associative skill memories

In recent years, research on movement primitives has gained increasing popularity. The original goals of movement primitives are based on the desire to have a sufficiently rich and abstract representation for movement generation, which allows for efficient teaching, trial-and-error learning, and generalization of motor skills (Schaal 1999). Thus, motor skills in robots should be acquired in a natural dialog with humans, e.g., by imitation learning and shaping, while skill refinement and generalization should be accomplished autonomously by the robot. Such a scenario resembles the way we teach children and connects to the bigger question of how the human brain accomplishes skill learning. In this paper, we review how a particular computational approach to movement primitives, called dynamic movement primitives, can contribute to learning motor skills. We will address imitation learning, generalization, trial-and-error learning by reinforcement learning, movement recognition, and control based on movement primitives. But we also want to go beyond the standard goals of movement primitives. The stereotypical movement generation with movement primitives entails predicting of sensory events in the environment. Indeed, all the sensory events associated with a movement primitive form an associative skill memory that has the potential of forming a most powerful representation of a complete motor skill.     

Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives

When describing robot motion with dynamic movement primitives (DMPs), goal (trajectory endpoint), shape and temporal scaling parameters are used. In reinforcement learning with DMPs, usually goals and temporal scaling parameters are predefined and only the weights for shaping a DMP are learned. Many tasks, however, exist where the best goal position is not a priori known, requiring to learn it. Thus, here we specifically address the question of how to simultaneously combine goal and shape parameter learning. This is a difficult problem because learning of both parameters could easily interfere in a destructive way. We apply value function approximation techniques for goal learning and direct policy search methods for shape learning. Specifically, we use “policy improvement with path integrals” and “natural actor critic” for the policy search. We solve a learning-to-pour-liquid task in simulations as well as using a Pa10 robot arm. Results for learning from scratch, learning initialized by human demonstration, as well as for modifying the tool for the learned DMPs are presented. We observe that the combination of goal and shape learning is stable and robust within large parameter regimes. Learning converges quickly even in the presence of disturbances, which makes this combined method suitable for robotic applications.     

基于改进动态运动基元的6D轨迹规划

动态运动基元(DMPs)轨迹规划方法可以简化机械臂控制中参数调整的复杂过程,快速生成运动轨迹,但是面对姿态的流形特性以及跨零点情况,现有的DMPs很难达到预期的效果.本文提出了一种基于改进DMPs的笛卡尔空间6D轨迹规划方法.该方法采用四元数描述姿态,实现了位置轨迹与姿态轨迹的无奇异表示.通过解耦强迫函数与起–终点状态差值项之间的关联,消除了跨零点引起的轨迹抖动、无法生成与翻转等问题.此外,基于机械臂和障碍物间的距离与偏角建立了虚拟阻抗关系,并将其耦合到动力学模型中,实现了机械臂末端的避障控制,避免了避障行为过早问题,有利于减少消耗.机械臂6D轨迹规划仿真和实验表明,本文提出的改进DMPs方法有效.     

Interaction learning for dynamic movement primitives used in cooperative robotic tasks

Since several years dynamic movement primitives (DMPs) are more and more getting into the center of interest for flexible movement control in robotics. In this study we introduce sensory feedback together with a predictive learning mechanism which allows tightly coupled dual-agent systems to learn an adaptive, sensor-driven interaction based on DMPs. The coupled conventional (no-sensors, no learning) DMP-system automatically equilibrates and can still be solved analytically allowing us to derive conditions for stability. When adding adaptive sensor control we can show that both agents learn to cooperate. Simulations as well as real-robot experiments are shown. Interestingly, all these mechanisms are entirely based on low level interactions without any planning or cognitive component.     

Probabilistic movement primitives under unknown system dynamics

Physical interaction requires robots to accurately follow kinematic trajectories while modulating the interaction forces to accomplish tasks and to be safe to the environment. However, current approaches rely on accurate physical models or iterative learning approaches. We present a versatile approach for physical interaction tasks, based on Movement Primitives (MPs) that can learn physical interaction tasks solely by demonstrations, without explicitly modeling the robot or the environment. We base our approach on the Probabilistic Movement Primitives (ProMPs), which utilizes the variance of the demonstrations to provide better generalization of the encoded skill, combine skills, and derive a controller that follows exactly the encoded trajectory distribution. However, the ProMP controller requires the system dynamics to be known. We present a reformulation of the ProMPs that allows accurate reproduction of the skill without modeling the system dynamics and, further, we extent our approach to incorporate external sensors, as for example, force/torque sensors. Our approach learns physical interaction tasks solely from demonstrations and online adapts the movement to force–torque sensor input. We derive a variable-stiffness controller in closed form that reproduces the trajectory distribution and the interaction forces present in the demonstrations. We evaluate our approach in simulated and real-robot tasks.     

手写运动的协作基元合成分析方法

研究一种生物运动神经控制机理与数据合成分析相结合的手写运动分析方法.特别地,将运动协作基元的概念用于手写运动数据分析,研究手写运动的协作基元合成分析方法,建立符合生物运动神经控制规律的手写运动数据理解模式.提出的协作基元合成分析过程由2个交替迭代的优化算法组成:其一,基于非负矩阵因子分解模式估计协作基元及调制幅度;其二,采用相似性最大化准则估计协作基元的激活时间.针对笔画切分的实验研究表明,采用协作基元合成分析方法获得的笔画切分结果,能够很好揭示相邻笔画之间的重叠连接模式,证实了所提方法的有效性.     

Trajectory generation and control of a knee exoskeleton based on dynamic movement primitives for sit-to-stand assistance

This paper presents a novel control approach for a knee exoskeleton to assist individuals with lower extremity weakness during sit-to-stand motion. The proposed method consists of a trajectory generator and an impedance controller. The trajectory generator uses a library of sample trajectories as the training data and the initial joint angles as the input to predict the user’s intended sit-to-stand trajectory. Utilizing the dynamic movement primitives theory, the trajectory generator represents the predicted trajectory in a time-normalized and rather a flexible framework. The impedance controller is then employed to provide assistance by guiding the knee joint to move along the predicted trajectory. Moreover, the human-exoskeleton interaction force is used as the feedback for on-line adaptation of the trajectory speed. The proposed control strategy was tested on a healthy adult who wore the knee exoskeleton on his leg. The subject was asked to perform a number of sit-to-stand movements from different sitting positions. Next, the measured data and the inverse dynamic model of the human-exoskeleton system are used to calculate the knee power and torque profiles. The results reveal that average muscle activity decreases when the subject is assisted by the exoskeleton.     

Reinforcement learning based on movement primitives for contact tasks

Recently, robot learning through deep reinforcement learning has incorporated various robot tasks through deep neural networks, without using specific control or recognition algorithms. However, this learning method is difficult to apply to the contact tasks of a robot, due to the exertion of excessive force from the random search process of reinforcement learning. Therefore, when applying reinforcement learning to contact tasks, solving the contact problem using an existing force controller is necessary. A neural-network-based movement primitive (NNMP) that generates a continuous trajectory which can be transmitted to the force controller and learned through a deep deterministic policy gradient (DDPG) algorithm is proposed for this study. In addition, an imitation learning algorithm suitable for NNMP is proposed such that the trajectories similar to the demonstration trajectory are stably generated. The performance of the proposed algorithms was verified using a square peg-in-hole assembly task with a tolerance of 0.1 mm. The results confirm that the complicated assembly trajectory can be learned stably through NNMP by the proposed imitation learning algorithm, and that the assembly trajectory is improved by learning the proposed NNMP through the DDPG algorithm.     

Localization of primitives using adaptive projections

An application of an active method in order to compute highly accurate 3D localization of point features from few projections is presented. The angle of projection of the image is controlled by the system and directed to extract 3D information from the environment in a manner leading to accurate location in less computation.This model is relevant for tomographic reconstruction, for feature based stereo and for model based robot registration.     

Computer-Assisted Learning using dynamic electronic books

While lectures and discussions provide a means of introducing a subject and inspiring further work, traditionally it has been the book that reinforces and expands that knowledge. This paper argues the case for the provision of a book-like presentation computer-assisted learning (CAL) system, manipulated with much the same ease as a real book. with built-in video/animation facilities, that might provide a more suitable environment, particularly in tertiary education, than that provided by conventional CAL systems. This paper describes a general piece of software called a book emulator that has been modified for, and applied to CAL in an attempt to reflect these attributes.     

A multiprocessor using protocol-based programming primitives

A general strategy is presented for multiprocessing that combines programming technique, machine architecture, and performance estimation. The programmer decomposes an application into manipulations of protocol-based programming primitives (protocols) usingPlans andscenarios from software engineering. The programmer may select from generic protocols, which include shared-memory locations and messages, or may build his own. A system architecture that supports efficient emlation of protocols is presented along with a method of estimating program performance based on network characteristics. Results are given from a protocol-based operating system on the 64 processor BTL Hypercube multiprocessor.     

Automatic segmentation and identification of mixed-language speech using delta-BIC and LSA-based GMMs

This paper proposes an approach to segmenting and identifying mixed-language speech. A delta Bayesian information criterion (delta-BIC) is firstly applied to segment the input speech utterance into a sequence of language-dependent segments using acoustic features. A VQ-based bi-gram model is used to characterize the acoustic-phonetic dynamics of two consecutive codewords in a language. Accordingly the language-specific acoustic-phonetic property of sequence of phones was integrated in the identification process. A Gaussian mixture model (GMM) is used to model codeword occurrence vectors orthonormally transformed using latent semantic analysis (LSA) for each language-dependent segment. A filtering method is used to smooth the hypothesized language sequence and thus eliminate noise-like components of the detected language sequence generated by the maximum likelihood estimation. Finally, a dynamic programming method is used to determine globally the language boundaries. Experimental results show that for Mandarin, English, and Taiwanese, a recall rate of 0.87 for language boundary segmentation was obtained. Based on this recall rate, the proposed approach achieved language identification accuracies of 92.1% and 74.9% for single-language and mixed-language speech, respectively.     

Efficient parallelizable hashing using small non-compressing primitives

A well-established method of constructing hash functions is to base them on non-compressing primitives, such as one-way functions or permutations. In this work, we present \(S^r\), an \(rn\)-to-\(n\)-bit compression function (for \(r\ge 1\)) making \(2r-1\) calls to \(n\)-to-\(n\)-bit primitives (random functions or permutations). \(S^r\) compresses its inputs at a rate (the amount of message blocks per primitive call) up to almost 1/2, and it outperforms all existing schemes with respect to rate and/or the size of underlying primitives. For instance, instantiated with the \(1600\)-bit permutation of NIST’s SHA-3 hash function standard, it offers about \(800\)-bit security at a rate of almost 1/2, while SHA-3-512 itself achieves only \(512\)-bit security at a rate of about \(1/3\). We prove that \(S^r\) achieves asymptotically optimal collision security against semi-adaptive adversaries up to almost \(2^{n/2}\) queries and that it can be made preimage secure up to \(2^n\) queries using a simple tweak.     

Approximation of n-dimensional data using spherical and ellipsoidal primitives

This paper discusses the problem of approximating data points in n-dimensional Euclidean space using spherical and ellipsoidal surfaces. A closed form solution is provided for spherical approximation, while an efficient, globally optimal solution for the ellipsoidal problem is proposed in terms of semidefinite programming. In addition, the paper presents a result for robust fitting in presence of outliers, and illustrates the theory with several numerical examples. A brief survey is also presented on the solutions to other relevant geometric approximation problems, such as ellipsoidal covering of convex hulls and pattern separation.     

Learning from partially supervised data using mixture models and belief functions

This paper addresses classification problems in which the class membership of training data are only partially known. Each learning sample is assumed to consist of a feature vector x_i∈X and an imprecise and/or uncertain “soft” label m_i defined as a Dempster-Shafer basic belief assignment over the set of classes. This framework thus generalizes many kinds of learning problems including supervised, unsupervised and semi-supervised learning. Here, it is assumed that the feature vectors are generated from a mixture model. Using the generalized Bayesian theorem, an extension of Bayes’ theorem in the belief function framework, we derive a criterion generalizing the likelihood function. A variant of the expectation maximization (EM) algorithm, dedicated to the optimization of this criterion is proposed, allowing us to compute estimates of model parameters. Experimental results demonstrate the ability of this approach to exploit partial information about class labels.     

RMR-efficient implementations of comparison primitives using read and write operations

We consider asynchronous multiprocessors where processes communicate only by reading or writing shared memory. We show how to implement consensus, compare-and-swap and other comparison primitives, as well as load-linked/store-conditional (LL/SC) using only a constant number of remote memory references (RMRs), in both the cache-coherent and the distributed-shared-memory models of such multiprocessors. Our implementations are blocking, rather than wait-free: they ensure progress provided all processes that invoke the implemented primitive are live. Our results imply that any algorithm using read and write operations, and either comparison primitives or LL/SC, can be simulated by an algorithm that uses read and write operations only, with at most a constant-factor increase in RMR complexity.     

Enabling control software generation by using mechatronics modeling primitives

Mechatronic systems are characterized by the synergetic integration of mechanic, electronic, software and control design aspects. The development of control software requires data and information from all design domains in order to create the required integrated functionality. This paper proposes a method that combines function modeling and multi-domain modeling primitives to generate control software automatically. An architecture model, based on the Function-Behavior-State modeling paradigm, provides the decomposition and flow of both functionality and implementation, which serves as input to a knowledge-based engineering application. The control software is subsequently extracted from a virtual product model composed of instantiated modeling primitives. A case study of a mobile robot shows how for a specific application the modeling are defined and how a high-level function model for an environment mapping mission is translated into directly implementable software code. This approach could be extended to real-life mechatronic products, and will improve consistency and reduce development time and cost.