Robotic grasping and manipulation through human visuomotor learning

A major goal of robotics research is to develop techniques that allow non-experts to teach robots dexterous skills. In this paper, we report our progress on the development of a framework which exploits human sensorimotor learning capability to address this aim. The idea is to place the human operator in the robot control loop where he/she can intuitively control the robot, and by practice, learn to perform the target task with the robot. Subsequently, by analyzing the robot control obtained by the human, it is possible to design a controller that allows the robot to autonomously perform the task. First, we introduce this framework with the ball-swapping task where a robot hand has to swap the position of the balls without dropping them, and present new analyses investigating the intrinsic dimension of the ball-swapping skill obtained through this framework. Then, we present new experiments toward obtaining an autonomous grasp controller on an anthropomorphic robot. In the experiments, the operator directly controls the (simulated) robot using visual feedback to achieve robust grasping with the robot. The data collected is then analyzed for inferring the grasping strategy discovered by the human operator. Finally, a method to generalize grasping actions using the collected data is presented, which allows the robot to autonomously generate grasping actions for different orientations of the target object.     

Methodology of Concept Control Synthesis to Avoid Unmoving and Moving Obstacles (II)

Dynamic path generation problem of robot in environment with other unmoving and moving objects is considered. Generally, the problem is known in literature as find path or robot motion planning. In this paper we apply the behavioral cloning approach to design the robot controller. In behavioral cloning, the system learns from control traces of a human operator. The task for the given problem is to find a controller not only in the form of the explicit mathematical expression. So RBF neural network is used also. The goal is to apply controller for the mobile robot motion planning in situation with infinite number of obstacles. The advantage of this approach lies in the fact that a complete path can be defined off-line, without using sophisticated symbolical models of obstacles.     

Human–Robot co-manipulation during surface tooling: A general framework based on impedance control,haptic rendering and discrete geometry

Despite the advancements in machine learning and artificial intelligence, there are many tooling tasks with cognitive aspects that are rather challenging for robots to handle in full autonomy, thus still requiring a certain degree of interaction with a human operator. In this paper, we propose a theoretical framework for both planning and execution of robot-surface contact tasks whereby interaction with a human operator can be accommodated to a variable degree.The starting point is the geometry of surface, which we assume known and available in a discretized format, e.g. through scanning technologies. To allow for realtime computation, rather than interacting with thousands of vertices, the robot only interacts with a single proxy, i.e. a massless virtual object constrained to ‘live on’ the surface and subject to first order viscous dynamics. The proxy and an impedance-controlled robot are then connected through tuneable and possibly viscoelastic coupling, i.e. (virtual) springs and dampers. On the one hand, the proxy slides along discrete geodesics of the surface in response to both viscoelastic coupling with the robot and to a possible external force (a virtual force which can be used to induce autonomous behaviours). On the other hand, the robot is free to move in 3D in reaction to the same viscoelastic coupling as well as to a possible external force, which includes an actual force exerted by a human operator. The proposed approach is multi-objective in the sense that different operational (autonomous/collaborative) and interactive (for contact/non-contact tasks) modalities can be realized by simply modulating the viscoelastic coupling as well as virtual and physical external forces. We believe that our proposed framework might lead to a more intuitive interfacing to robot programming, as opposed to standard coding. To this end, we also present numerical and experimental studies demonstrating path planning as well as autonomous and collaborative interaction for contact tasks with a free-form surface.     

Dyna-Q-based vector direction for path planning problem of autonomous mobile robots in unknown environments

Reinforcement learning (RL) is a popular method for solving the path planning problem of autonomous mobile robots in unknown environments. However, the primary difficulty faced by learning robots using the RL method is that they learn too slowly in obstacle-dense environments. To more efficiently solve the path planning problem of autonomous mobile robots in such environments, this paper presents a novel approach in which the robot’s learning process is divided into two phases. The first one is to accelerate the learning process for obtaining an optimal policy by developing the well-known Dyna-Q algorithm that trains the robot in learning actions for avoiding obstacles when following the vector direction. In this phase, the robot’s position is represented as a uniform grid. At each time step, the robot performs an action to move to one of its eight adjacent cells, so the path obtained from the optimal policy may be longer than the true shortest path. The second one is to train the robot in learning a collision-free smooth path for decreasing the number of the heading changes of the robot. The simulation results show that the proposed approach is efficient for the path planning problem of autonomous mobile robots in unknown environments with dense obstacles.     

基于离散灰狼算法的喷涂机器人路径规划方法

针对目前用于复杂结构实体喷涂的机器人路径规划方法存在的效率低、未考虑碰撞以及适用性差等问题,提出一种用于求解多层决策问题的离散灰狼算法,并把该算法用于该路径规划问题的求解。为了将连续域灰狼算法改为用于求解多层决策问题的离散灰狼算法,采用矩阵编码方法解决多层决策问题的编码问题,提出基于先验知识与随机选择的混合初始化方法提高算法求解效率和精度,运用交叉算子与两级变异算子定义离散域灰狼算法的种群更新策略。另外,运用图论将喷涂机器人路径规划问题简化为广义旅行商问题,并建立了该问题的最短路径模型和路径碰撞模型。在路径规划实验中,相较于粒子群算法、遗传算法和蚁群算法,提出的算法规划的平均路径长度分别减小了5.0%、5.5%和6.6%,碰撞次数降低为0,且路径更平滑。实验结果表明,提出的算法能够有效提高喷涂机器人的喷涂效率,以及喷涂路径的安全性和适用性。     

Methodology of Concept Control Synthesis to Avoid Unmoving and Moving Obstacles

The dynamic path generation problem of robots in environments with other unmoving and moving objects is considered. Generally, the problem is known in the literature as find path or robot motion planning. In this paper, we apply the behavioral cloning approach to design the robot controller. In behavioral cloning, the system learns from control traces of a human operator. The task for the given problem is to find a controller in the form of an explicit mathematical expression. Thus, machine learning programs to induce the operator's trajectories as a set of symbolic constraints are used. Then, mathematical induction to generalize the obtained equations in order to apply them in situ with an infinite number of obstacles is also used. A method to evaluate cloning success is proposed. The typical kind of noise is included.     

Physical simulation of land vehicles with obstacle avoidance and various terrain interactions

Programming the motions of an autonomous planetary robot moving in an hostile and hazardous environment is a complex task which requires both the construction of nominal motion plans and the anticipation as far as possible of the effects of the interactions existing between the vehicle and the terrain. In this paper we show how physical models and dynamic simulation tools can be used for amending and completing a nominal motion plan provided by a classical geometrical path planner. The purpose of our physical modeller-simulator is to anticipate the dynamic behaviour of the vehicle while executing the nominal motion plan. Then the obtained simulation results can be used to assess and optimize the nominal motion plan. In the first part, we outline the physical models that have been used for modelling the different types of vehicle, of terrain and of vehicle-surface interactions. Then we formulate the motion planning problem through the definition of two basic abstract constructions derived from physical model: the concept of generalized obstacle and the concept of physical target. We show with various examples how it is possible, when using this method, to solve the locomotion problem and the obstacle avoidance problem simultaneously and, furthermore, to provide the human operator with a true force feedback gestural control over the simulated robot.     

基于离散灰狼算法的喷涂机器人路径规划方法

针对目前用于复杂结构实体喷涂的机器人路径规划方法存在的效率低、未考虑碰撞以及适用性差等问题，提出一种用于求解多层决策问题的离散灰狼算法，并把该算法用于该路径规划问题的求解。为了将连续域灰狼算法改为用于求解多层决策问题的离散灰狼算法，采用矩阵编码方法解决多层决策问题的编码问题，提出基于先验知识与随机选择的混合初始化方法提高算法求解效率和精度，运用交叉算子与两级变异算子定义离散域灰狼算法的种群更新策略。另外，运用图论将喷涂机器人路径规划问题简化为广义旅行商问题，并建立了该问题的最短路径模型和路径碰撞模型。在路径规划实验中，相较于粒子群算法、遗传算法和蚁群算法，提出的算法规划的平均路径长度分别减小了5.0%、5.5%和6.6%，碰撞次数降低为0，且路径更平滑。实验结果表明，提出的算法能够有效提高喷涂机器人的喷涂效率，以及喷涂路径的安全性和适用性。     

A visual tool for computer supported learning: The robot motion planning example

We introduce an effective computer aided learning visual tool (CALVT) to teach graph-based applications. We present the robot motion planning problem as an example of such applications. The proposed tool can be used to simulate and/or further to implement practical systems in different areas of computer science such as graphics, computational geometry, robotics and networking. In the robot motion planning example, CALVT enables users to setup the working environment by creating obstacles and a robot of different shapes, specifying starting and goal positions, and setting other path or environment parameters from a user-friendly interface. The path planning system involves several phases. Each of these modules is complex and therefore we provide the possibility of visualizing graphically the output of each phase. Based on our experience, this tool has been an effective one in classroom teaching. It not only cuts down, significantly, on the instructor’s time and effort but also motivates senior/graduate students to pursue work in this specific area of research.     

Real-time humanoid whole-body remote control framework for imitating human motion based on kinematic mapping and motion constraints

In this paper, we propose a whole-body remote control framework that enables a robot to imitate human motion efficiently. The framework is divided into kinematic mapping and quadratic programming based whole-body inverse kinematics. In the kinematic mapping, the human motion obtained through a data acquisition device is transformed into a reference motion that is suitable for the robot to follow. To address differences in the kinematic configuration and dynamic properties of the robot and human, quadratic programming is used to calculate the joint angles of the robot considering self-collision, joint limits, and dynamic stability. To address dynamic stability, we use constraints based on the divergent component of motion and zero moment point in the linear inverted pendulum model. Simulation using Choreonoid and a locomotion experiment using the HUBO2+ demonstrate the performance of the proposed framework. The proposed framework has the potential to reduce the preview time or offline task computation time found in previous approaches and hence improve the similarity of human and robot motion while maintaining stability.     

Efficient behavior learning in human–robot collaboration

We present a novel method for a robot to interactively learn, while executing, a joint human–robot task. We consider collaborative tasks realized by a team of a human operator and a robot helper that adapts to the human’s task execution preferences. Different human operators can have different abilities, experiences, and personal preferences so that a particular allocation of activities in the team is preferred over another. Our main goal is to have the robot learn the task and the preferences of the user to provide a more efficient and acceptable joint task execution. We cast concurrent multi-agent collaboration as a semi-Markov decision process and show how to model the team behavior and learn the expected robot behavior. We further propose an interactive learning framework and we evaluate it both in simulation and on a real robotic setup to show the system can effectively learn and adapt to human expectations.     

基于总体势减小的动态调度技术解决多机器人的路径规划

本文提出了一种解决多机器人路径规划与协调问题的新方法：基于总体势减小的优 先级动态调度策略．文中引入了总体势的概念,机器人从起始点向目标点运动过程中,始终 沿着总体势减小的方向进行,逐步引导机器人导航任务的完成．     

Automatic Viewpoint Planning in Teleoperation of a Mobile Robot

Camera viewpoint has significant impact on operators situation awareness in teleoperation. This paper presents a method for automatic optimal positioning of a single camera for a remotely navigated mobile robot in systems with a controllable camera platform. The proposed algorithm continuously adjusts the camera view of the workspace based on the task circumstances, allowing the operator to focus mainly on navigation and manipulation. The workspace and motion limits of the camera system and the location of the obstacles are taken into consideration in the camera view planning by formulating and solving a constrained optimization problem in real-time. A head tracking system enables the operator to use his/her head movements as an extra control input to guide the camera placement, if and when necessary. The proposed viewpoint control framework has been implemented and evaluated in a teleoperation experiment with a mobile robot. Results of a user study comparing this approach to two other common viewpoint control strategies are also reported.     

Robot motion design using bunraku emotional expressions – focusing on Jo-Ha-Kyū in sounds and movements*

One of the UNESCO intangible cultural heritages Bunraku puppets can play one of the most beautiful puppet motions in the world. The Bunraku puppet motions can express emotions without the so-called ‘Uncanny Valley.’ We try to convert these emotional motions into robot affective motions so that robots can interact with human beings more comfortable. In so doing, in the present paper, we present a robot motion design framework using Bunraku affective motions that are based on the so-called ‘Jo-Ha-Kyū,’ and convert a few simple Bunraku motions into a robot motions using one of deep learning methods. Our primitive experiments show that Jo-Ha-Kyū can be incorporated into robot motion design smoothly, and some simple affective robot motions can be designed using our proposed framework.     

Construction of the best ellipsoidal approximation of the attraction domain in stabilization problem for a wheeled robot

The synthesis control problem for the plane motion of a wheeled robot with constrained control resource is studied. The goal of the control is to bring the robot to an assigned curvilinear trajectory and to stabilize its motion along it. For a synthesized control law, the problem of finding the best in the sense of volume ellipsoidal approximation of the attraction domain of the target path is posed. To take into account constraints on the control, an approach based on methods of absolute stability theory is used, in the framework of which construction of an approximating ellipsoid reduces to solving a system of linear matrix inequalities. It is shown that the desired maximum-volume approximating ellipsoid can be found by solving a standard constrained optimization problem for a function of two variables.     

LEARNING TO SOLVE PROBLEMS FROM EXERCISES

It is a common observation that learning easier skills makes it possible to learn the more difficult skills. This fact is routinely exploited by parents, teachers, textbook writers, and coaches. From driving, to music, to science, there hardly exists a complex skill that is not learned by gradations. Natarajan's model of “learning from exercises” captures this kind of learning of efficient problem solving skills using practice problems or exercises ( Natarajan 1989 ). The exercises are intermediate subproblems that occur in solving the main problems and span all levels of difficulty. The learner iteratively bootstraps what is learned from simpler exercises to generalize techniques for solving more complex exercises. In this paper, we extend Natarajan's framework to the problem reduction setting where problems are solved by reducing them to simpler problems. We theoretically characterize the conditions under which efficient learning from exercises is possible. We demonstrate the generality of our framework with successful implementations in the Eight Puzzle, symbolic integration, and simulated robot planning domains illustrating three different representations of control knowledge, namely, macro‐operators, control rules, and decision lists. The results show that the learning rates for the exercises framework are competitive with those for learning from problems solved by the teacher.     

Learning to Walk by Imitation in Low-Dimensional Subspaces

In this paper, we provide the first demonstration that a humanoid robot can learn to walk directly by imitating a human gait obtained from motion capture (mocap) data without any prior information of its dynamics model. Programming a humanoid robot to perform an action (such as walking) that takes into account the robot's complex dynamics is a challenging problem. Traditional approaches typically require highly accurate prior knowledge of the robot's dynamics and environment in order to devise complex (and often brittle) control algorithms for generating a stable dynamic motion. Training using human mocap is an intuitive and flexible approach to programming a robot, but direct usage of mocap data usually results in dynamically unstable motion. Furthermore, optimization using high-dimensional mocap data in the humanoid full-body joint space is typically intractable. We propose a new approach to tractable imitation-based learning in humanoids without a robot's dynamic model. We represent kinematic information from human mocap in a low-dimensional subspace and map motor commands in this low-dimensional space to sensory feedback to learn a predictive dynamic model. This model is used within an optimization framework to estimate optimal motor commands that satisfy the initial kinematic constraints as best as possible while generating dynamically stable motion. We demonstrate the viability of our approach by providing examples of dynamically stable walking learned from mocap data using both a simulator and a real humanoid robot.     

Motion planning of skillful motions in assembly process through human demonstration

ABSTRACT

Skillful motions in the actual assembly process are challenging for the robot to generate with conventional motion planning approaches because some states during the human assembly can be too skillful to realize automatically due to the narrow passage. To deal with this problem, this paper develops a motion planning method using the human demonstration, which can be applied to complete skillful motions in the robotic assembly process. To demonstrate conveniently without redundant third-party devices, we attach augmented reality (AR) markers to the manipulated object to track and capture its poses during the human demonstration. To overcome the problem brought by the coarse resolution of the vision system, we extract the most important key poses from the demonstration data and employ them as clues to execute motion planning to suit the target precise task. As for the selection of key poses, two policies are compared, where the first and the second derivative of the main changing parameter of every key pose serve as criteria to determine the priority of utilizing key poses. Besides, a solution to deal with colliding key poses is also proposed. The effectiveness of the presented method is verified through some simulation examples and actual robot experiments.     

基于改进蝙蝠算法和三次样条插值的机器人路径规划

为更好地解决移动机器人路径规划问题, 改进蝙蝠算法的寻优性能, 拓展其应用领域, 提出了一种具有反向学习和正切随机探索机制的蝙蝠算法. 在全局搜索阶段的位置更新中引入动态扰动系数, 提高算法全局搜索能力; 在局部搜索阶段, 融入正切随机探索机制, 增强算法局部寻优的策略性, 避免算法陷入局部极值. 同时, 加入反向学习选择策略, 进一步平衡蝙蝠种群多样性和算法局部开采能力, 提高算法的收敛精度. 然后, 把改进算法与三次样条插值方法相结合去求解机器人全局路径规划问题, 定义了基于路径结点的编码方式, 构造了绕避障碍求解最短路径的方法和适应度函数. 最后, 在简单和复杂障碍环境下分别对单机器人和多机器人系统进行了路径规划对比实验. 实验结果表明, 改进后算法无论在最优解还是平均解方面都要优于其他几种对比算法, 对于求解机器人全局路径规划问题具有较好的可行性和有效性.