A cognitive framework for imitation learning

In order to have a robotic system able to effectively learn by imitation, and not merely reproduce the movements of a human teacher, the system should have the capabilities of deeply understanding the perceived actions to be imitated. This paper deals with the development of cognitive architecture for learning by imitation in which a rich conceptual representation of the observed actions is built. The purpose of the following discussion is to show how this Conceptual Area can be employed to efficiently organize perceptual data, to learn movement primitives from human demonstration and to generate complex actions by combining and sequencing simpler ones. The proposed architecture has been tested on a robotic system composed of a PUMA 200 industrial manipulator and an anthropomorphic robotic hand.     

From dynamic movement primitives to associative skill memories

In recent years, research on movement primitives has gained increasing popularity. The original goals of movement primitives are based on the desire to have a sufficiently rich and abstract representation for movement generation, which allows for efficient teaching, trial-and-error learning, and generalization of motor skills (Schaal 1999). Thus, motor skills in robots should be acquired in a natural dialog with humans, e.g., by imitation learning and shaping, while skill refinement and generalization should be accomplished autonomously by the robot. Such a scenario resembles the way we teach children and connects to the bigger question of how the human brain accomplishes skill learning. In this paper, we review how a particular computational approach to movement primitives, called dynamic movement primitives, can contribute to learning motor skills. We will address imitation learning, generalization, trial-and-error learning by reinforcement learning, movement recognition, and control based on movement primitives. But we also want to go beyond the standard goals of movement primitives. The stereotypical movement generation with movement primitives entails predicting of sensory events in the environment. Indeed, all the sensory events associated with a movement primitive form an associative skill memory that has the potential of forming a most powerful representation of a complete motor skill.     

Fuzzy clustering of human motor motion

Acquisition of the behavioural skills of a human operator and recreating them in an intelligent autonomous system has been a critical but rather challenging step in the development of complex intelligent autonomous systems. Development of a systematic and generic method for realising this process by acquiring human postural and motor movements is explored. This is achieved by breaking down the human motion into a number of segments called motion or skill primitives. The proposed methodology is developed based on studying the movement of the human hand. The motion is measured by a dual-axis accelerometer and a gyroscope mounted on the hand. The gyroscope locates the position and configuration of the hand, whereas the accelerometer measures the kinematics parameters of the movement. The covariance and the mean of the data produced by the sensors are used as features in the clustering process. A fuzzy clustering method is developed and applied to identify different movements of the human hand. The proposed clustering approach identifies the sequence of the motion primitives embedded in the data produced from the human wrist movement. A review of the previous work in the area is carried out and the developed methodology is described. An overview of the experimental setup and procedures to validate the approach is given. The results of the validation are analysed critically and some conclusions are drawn.     

机器人运动轨迹的模仿学习综述EI北大核心CSCD

作为机器人技能学习中的一个重要分支,模仿学习近年来在机器人系统中得到了广泛的应用.模仿学习能够将人类的技能以一种相对直接的方式迁移到机器人系统中,其思路是先从少量示教样本中提取相应的运动特征,然后将该特征泛化到新的情形.本文针对机器人运动轨迹的模仿学习进行综述.首先详细解释模仿学习中的技能泛化、收敛性和外插等基本问题;其次从原理上对动态运动基元、概率运动基元和核化运动基元等主要的模仿学习算法进行介绍;然后深入地讨论模仿学习中姿态和刚度矩阵的学习问题、协同和不确定性预测的问题以及人机交互中的模仿学习等若干关键问题;最后本文探讨了结合因果推理的模仿学习等几个未来的发展方向.     

Movement primitives as a robotic tool to interpret trajectories through learning-by-doing

Articulated movements are fundamental in many human and robotic tasks.While humans can learn and generalise arbitrarily long sequences of movements,and particularly can optimise them to ft the constraints and features of their body,robots are often programmed to execute point-to-point precise but fxed patterns.This study proposes a new approach to interpreting and reproducing articulated and complex trajectories as a set of known robot-based primitives.Instead of achieving accurate reproductions,the proposed approach aims at interpreting data in an agent-centred fashion,according to an agent s primitive movements.The method improves the accuracy of a reproduction with an incremental process that seeks frst a rough approximation by capturing the most essential features of a demonstrated trajectory.Observing the discrepancy between the demonstrated and reproduced trajectories,the process then proceeds with incremental decompositions and new searches in sub-optimal parts of the trajectory.The aim is to achieve an agent-centred interpretation and progressive learning that fts in the frst place the robots capability,as opposed to a data-centred decomposition analysis.Tests on both geometric and human generated trajectories reveal that the use of own primitives results in remarkable robustness and generalisation properties of the method.In particular,because trajectories are understood and abstracted by means of agent-optimised primitives,the method has two main features： 1） Reproduced trajectories are general and represent an abstraction of the data.2） The algorithm is capable of reconstructing highly noisy or corrupted data without pre-processing thanks to an implicit and emergent noise suppression and feature detection.This study suggests a novel bio-inspired approach to interpreting,learning and reproducing articulated movements and trajectories.Possible applications include drawing,writing,movement generation,object manipulation,and other tasks where the performance requires human-like interpretation and generalisation capabilities.     

Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning

In this paper we present a robot control architecture for learning by imitation which takes inspiration from recent discoveries in action observation/execution experiments with humans and other primates. The architecture implements two basic processing principles: (1) imitation is primarily directed toward reproducing the outcome of an observed action sequence rather than reproducing the exact action means, and (2) the required capacity to understand the motor intention of another agent is based on motor simulation. The control architecture is validated in a robot system imitating in a goal-directed manner a grasping and placing sequence displayed by a human model. During imitation, skill transfer occurs by learning and representing appropriate goal-directed sequences of motor primitives. The robustness of the goal-directed organization of the controller is tested in the presence of incomplete visual information and changes in environmental constraints.     

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.     

FORMS: A flexible object recognition and modelling system

We describe a flexible object recognition and modelling system (FORMS) which represents and recognizes animate objects from their silhouettes. This consists of a model for generating the shapes of animate objects which gives a formalism for solving the inverse problem of object recognition. We model all objects at three levels of complexity: (i) the primitives, (ii) the mid-grained shapes, which are deformations of the primitives, and (iii) objects constructed by using a grammar to join mid-grained shapes together. The deformations of the primitives can be characterized by principal component analysis or modal analysis. When doing recognition the representations of these objects are obtained in a bottom-up manner from their silhouettes by a novel method for skeleton extraction and part segmentation based on deformable circles. These representations are then matched to a database of prototypical objects to obtain a set of candidate interpretations. These interpretations are verified in a top-down process. The system is demonstrated to be stable in the presence of noise, the absence of parts, the presence of additional parts, and considerable variations in articulation and viewpoint. Finally, we describe how such a representation scheme can be automatically learnt from examples.     

Data visualization optimization via computational modeling of perception

We present a method for automatically evaluating and optimizing visualizations using a computational model of human vision. The method relies on a neural network simulation of early perceptual processing in the retina and primary visual cortex. The neural activity resulting from viewing flow visualizations is simulated and evaluated to produce a metric of visualization effectiveness. Visualization optimization is achieved by applying this effectiveness metric as the utility function in a hill-climbing algorithm. We apply this method to the evaluation and optimization of 2D flow visualizations, using two visualization parameterizations: streaklet-based and pixel-based. An emergent property of the streaklet-based optimization is head-to-tail streaklet alignment. It had been previously hypothesized the effectiveness of head-to-tail alignment results from the perceptual processing of the visual system, but this theory had not been computationally modeled. A second optimization using a pixel-based parameterization resulted in a LIC-like result. The implications in terms of the selection of primitives is discussed. We argue that computational models can be used for optimizing complex visualizations. In addition, we argue that they can provide a means of computationally evaluating perceptual theories of visualization, and as a method for quality control of display methods.     

Imitating others by composition of primitive actions: A neuro-dynamic model

This paper introduces a novel neuro-dynamical model that accounts for possible mechanisms of action imitation and learning. It is considered that imitation learning requires at least two classes of generalization. One is generalization over sensory–motor trajectory variances, and the other class is on cognitive level which concerns on more qualitative understanding of compositional actions by own and others which do not necessarily depend on exact trajectories. This paper describes a possible model dealing with these classes of generalization by focusing on the problem of action compositionality. The model was evaluated in the experiments using a small humanoid robot. The robot was trained with a set of different actions concerning object manipulations which can be decomposed into sequences of action primitives. Then the robot was asked to imitate a novel compositional action demonstrated by a human subject which are composed from prior-learned action primitives. The results showed that the novel action can be successfully imitated by decomposing and composing it with the primitives by means of organizing unified intentional representation hosted by mirror neurons even though the trajectory-level appearance is different between the ones of observed and those of self-generated.     

Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory

We describe a general framework for learning perception-based navigational behaviors in autonomous mobile robots. A hierarchical behavior-based decomposition of the control architecture is used to facilitate efficient modular learning. Lower level reactive behaviors such as collision detection and obstacle avoidance are learned using a stochastic hill-climbing method while higher level goal-directed navigation is achieved using a self-organizing sparse distributed memory. The memory is initially trained by teleoperating the robot on a small number of paths within a given domain of interest. During training, the vectors in the sensory space as well as the motor space are continually adapted using a form of competitive learning to yield basis vectors that efficiently span the sensorimotor space. After training, the robot navigates from arbitrary locations to a desired goal location using motor output vectors computed by a saliency-based weighted averaging scheme. The pervasive problem of perceptual aliasing in finite-order Markovian environments is handled by allowing both current as well as the set of immediately preceding perceptual inputs to predict the motor output vector for the current time instant. We describe experimental and simulation results obtained using a mobile robot equipped with bump sensors, photosensors and infrared receivers, navigating within an enclosed obstacle-ridden arena. The results indicate that the method performs successfully in a number of navigational tasks exhibiting varying degrees of perceptual aliasing.     

Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory

We describe a general framework for learning perception-based navigational behaviors in autonomous mobile robots. A hierarchical behavior-based decomposition of the control architecture is used to facilitate efficient modular learning. Lower level reactive behaviors such as collision detection and obstacle avoidance are learned using a stochastic hill-climbing method while higher level goal-directed navigation is achieved using a self-organizing sparse distributed memory. The memory is initially trained by teleoperating the robot on a small number of paths within a given domain of interest. During training, the vectors in the sensory space as well as the motor space are continually adapted using a form of competitive learning to yield basis vectors that efficiently span the sensorimotor space. After training, the robot navigates from arbitrary locations to a desired goal location using motor output vectors computed by a saliency-based weighted averaging scheme. The pervasive problem of perceptual aliasing in finite-order Markovian environments is handled by allowing both current as well as the set of immediately preceding perceptual inputs to predict the motor output vector for the current time instant. We describe experimental and simulation results obtained using a mobile robot equipped with bump sensors, photosensors and infrared receivers, navigating within an enclosed obstacle-ridden arena. The results indicate that the method performs successfully in a number of navigational tasks exhibiting varying degrees of perceptual aliasing.     

Policy search for motor primitives in robotics

Many motor skills in humanoid robotics can be learned using parametrized motor primitives. While successful applications to date have been achieved with imitation learning, most of the interesting motor learning problems are high-dimensional reinforcement learning problems. These problems are often beyond the reach of current reinforcement learning methods. In this paper, we study parametrized policy search methods and apply these to benchmark problems of motor primitive learning in robotics. We show that many well-known parametrized policy search methods can be derived from a general, common framework. This framework yields both policy gradient methods and expectation-maximization (EM) inspired algorithms. We introduce a novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives. We compare this algorithm, both in simulation and on a real robot, to several well-known parametrized policy search methods such as episodic REINFORCE, ??Vanilla?? Policy Gradients with optimal baselines, episodic Natural Actor Critic, and episodic Reward-Weighted Regression. We show that the proposed method out-performs them on an empirical benchmark of learning dynamical system motor primitives both in simulation and on a real robot. We apply it in the context of motor learning and show that it can learn a complex Ball-in-a-Cup task on a real Barrett WAM? robot arm.     

绕线式异步电动机转子串电阻分级起动过程的仿真

用计算机对绕线式异步电动机转子串电阻的分级起动过程进行仿真，使起动器的设计和选用更为快捷合理。该文用MATLAB软件中的电气系统模块库建立了绕线式异步电动机转子串电阻分级起动的瞬态仿真模型。其中，起动器的各级起动电阻的数值是根据异步电动机的T型等效电路对应的电流方程，转矩方程，用数值方法通过优化计算确定的，断路器的闭合时间是根据系统的运动方程用数值分计算确定的，最后通过一个实例对电机的启动过程进行仿真并给出结果。     

A New Algorithm and System for the Characterization of Handwriting Strokes with Delta-Lognormal Parameters

In this paper, we present a new analytical method for estimating the parameters of delta-lognormal functions and characterizing handwriting strokes. According to the kinematic theory of rapid human movements, these parameters contain information on both the motor commands and the timing properties of a neuromuscular system. The new algorithm, called XZERO, exploits relationships between the zero crossings of the first and second time derivatives of a lognormal function and its four basic parameters. The methodology is described and then evaluated under various testing conditions. The new tool allows a greater variety of stroke patterns to be processed automatically. Furthermore, for the first time, the extraction accuracy is quantified empirically, taking advantage of the exponential relationships that link the dispersion of the extraction errors with its signal-to-noise ratio. A new extraction system which combines this algorithm with two other previously published methods is also described and evaluated. This system provides researchers involved in various domains of pattern analysis and artificial intelligence with new tools for the basic study of single strokes as primitives for understanding rapid human movements.     

Perceptual primitives from an extended 4D Hough transform

Directional features extracted from Gabor wavelets responses were used to train a structure of self-organising maps, thus classifying each pixel in the image within a neuron-map. Resulting directional primitives were grouped into perceptual primitives introducing an extended 4D Hough transform to group pixels with similar directional features. These can then be used as perceptual primitives to detect salient structures. The proposed method has independently fixed parameters that do not need to be tuned for different kind or quality of images. We present results in application to noisy FLIR images and show that line primitives for complex structures, such as bridges, or simple structures, such as runways, can be found by this approach. We compare and demonstrate the quality of our results with those obtained through a parameter-dependent traditional Canny edge detector and Hough line finding process.