Learning robotic hand-eye coordination through a developmental constraint driven approach

The skill of robotic hand-eye coordination not only helps robots to deal with real time environment,but also afects the fundamental framework of robotic cognition.A number of approaches have been developed in the literature for construction of the robotic hand-eye coordination.However,several important features within infant developmental procedure have not been introduced into such approaches.This paper proposes a new method for robotic hand-eye coordination by imitating the developmental progress of human infants.The work employs a brain-like neural network system inspired by infant brain structure to learn hand-eye coordination,and adopts a developmental mechanism from psychology to drive the robot.The entire learning procedure is driven by developmental constraint: The robot starts to act under fully constrained conditions,when the robot learning system becomes stable,a new constraint is assigned to the robot.After that,the robot needs to act with this new condition again.When all the contained conditions have been overcome,the robot is able to obtain hand-eye coordination ability.The work is supported by experimental evaluation,which shows that the new approach is able to drive the robot to learn autonomously,and make the robot also exhibit developmental progress similar to human infants.     

Human direct teaching of industrial articulated robot arms based on force-free control

Force-free control produces motion in a robot arm as if it were under conditions with no gravity and no friction. In this study, a method of force-free control is proposed for industrial articulated robot arms. The force-free control proposed was applied to the direct teaching of industrial articulated robot arms in that the robot arm was moved by direct human force. Generally, the teaching of industrial articulated robot arms is carried out using operational equipment called a teach-pendant. Smooth teaching can be achieved if direct teaching is applicable. The force-free control proposed enables humans to teach industrial articulated robot arms directly. The effectiveness of force-free control was confirmed by experimental work on an articulated robot arm with two degrees of freedom. This work was presented in part at the Fifth International Symposium on Artificial Life and Robotics, Oita, Japan, January 26–28, 2000     

基于目标检测的机器人手眼标定方法

智能协作机器人依赖视觉系统感知未知环境中的动态工作空间定位目标,实现机械臂对目标对象的自主抓取回收作业。RGB-D相机可采集场景中的彩色图和深度图,获取视野内任意目标三维点云,辅助智能协作机器人感知周围环境。为获取抓取机器人与RGB-D相机坐标系之间的转换关系,提出基于yolov3目标检测神经网络的机器人手眼标定方法。将3D打印球作为标靶球夹持在机械手末端,使用改进的yolov3目标检测神经网络实时定位标定球的球心,计算机械手末端中心在相机坐标系下的3D位置,同时运用奇异值分解方法求解机器人与相机坐标系转换矩阵的最小二乘解。在6自由度UR5机械臂和Intel RealSense D415深度相机上的实验结果表明,该标定方法无需辅助设备,转换后的空间点位置误差在2 mm以内,能较好满足一般视觉伺服智能机器人的抓取作业要求。     

Robot calibration by mobile camera systems

A measurement technique for kinematic calibration of robot manipulators, which uses a stereo hand-eye system with moving camera coordinates, is presented in this article. The calibration system consists of a pair of cameras rigidly mounted on the robot end-effector, a camera calibration board, and a robot calibration fixture. The stereo cameras are precalibrated using the camera calibration board so that the 3D coordinates of any object point seen by the stereo cameras can be computed with respect to the camera coordinate frame [C] defined by the calibration board. Because [C] is fixed with respect to the tool frame [T] of the robot, it moves with the robot hand from one calibration measurement configuration to another. On each face of the robot calibration fixture that defines the world coordinate frame [W], there are evenly spaced dot patterns of uniform shape. Each pattern defines a coordinate frame [E_i], whose pose is known in [W]. The dot pattern is designed in such a way that from a pair of images of the pattern, the pose of [E_i] can be estimated with respect to [C] in each robot calibration measurement. By that means the pose of [C] becomes known in [W] at each robot measurement configuration. For a sufficient number of measurement configurations, the homogeneous transformation from [W] to [C] (or equivalently to [T]), and thus the link parameters of the robot, can be identified using the least-squares techniques. Because the cameras perform local measurements only, the field-of-view of the camera system can be as small as 50 × 50 mm², resulting in an overall accuracy of the measurement system as high as 0.05 mm. This is at least 20 times better than the accuracy provided by vision-based measurement systems with a fixed camera coordinate frame using common off-the-shelf cameras. © 1994 John Wiley & Sons, Inc.     

Visual–Motor Coordination Using a Quantum Clustering Based Neural Control Scheme

Visual–Motor Coordination is a problem considered analogous to the hand-eye coordination in biological systems. In this work we propose a novel approach to this problem using Quantum Clustering and an extended Kohonen's Self-Organizing Feature Map (K-SOFM). This facilities the use of the method in varying workspaces by considering the joint angles of the robot arm. Unlike previous work, where a fixed topology for the input space is considered, the proposed approach determines a topology as the workspace varies. Quantum Clustering is a method which constructs a scale-space probability function and uses the Schroedinger equation and its lowest eigenstate to obtain a potential whose minimum gives the cluster centers. It transforms the input space into a Hilbert space, where it searches for its minimum. The motivation of this work is to identify the implicit relationship existing between the end-effector positions and the joint angles through Quantum Clustering and Neural Network methods to fine-tune the system to correctly identify the mapping.     

一种实用的手眼视觉系统定位方法

深入研究了机器人手眼视觉系统的立体定位问题。首先,重新将手眼问题公式化,得到一个新的系统模型;然后,在此基础上提出了一种实用有效的标定方法。其核心思想是直接将图像坐标映射到机器人基坐标系中,把系统参数作为一个整体来获取,而不必分别计算摄像机内部的每一个参数。与原方法相比,本方法可随意改变末端姿态定位,即定位时摄像机对目标取像的姿态不受任何约束。实验表明,该方法操作方便、实现简单、定位精度高。这一方法的提出克服了原方法的局限性,大大推广了手眼视觉系统的应用范围。     

Robust hand-eye calibration of 2D laser sensors using a single-plane calibration artefact

When a vision sensor is used in conjunction with a robot, hand-eye calibration is necessary to determine the accurate position of the sensor relative to the robot. This is necessary to allow data from the vision sensor to be defined in the robot's global coordinate system. For 2D laser line sensors hand-eye calibration is a challenging process because they only collect data in two dimensions. This leads to the use of complex calibration artefacts and requires multiple measurements be collected, using a range of robot positions. This paper presents a simple and robust hand-eye calibration strategy that requires minimal user interaction and makes use of a single planar calibration artefact. A significant benefit of the strategy is that it uses a low-cost, simple and easily manufactured artefact; however, the lower complexity can lead to lower variation in calibration data. In order to achieve a robust hand-eye calibration using this artefact, the impact of robot positioning strategies is considered to maintain variation. A theoretical basis for the necessary sources of input variation is defined by a mathematical analysis of the system of equations for the calibration process. From this, a novel strategy is specified to maximize data variation by using a circular array of target scan lines to define a full set of required robot positions. A simulation approach is used to further investigate and optimise the impact of robot position on the calibration process, and the resulting optimal robot positions are then experimentally validated for a real robot mounted laser line sensor. Using the proposed optimum method, a semi-automatic calibration process, which requires only four manually scanned lines, is defined and experimentally demonstrated.     

基于EAIDK的智能煤矸分拣系统设计

现有基于图像识别的煤矸石分拣方法实时性较差且整体分拣准确率不高,而基于密度的分拣方法适用于井下初选,成本较高。针对上述问题,设计实现了一种基于EAIDK的智能煤矸分拣系统。采用嵌入式人工智能开发平台EAIDK构建矸石识别和分拣控制硬件平台,在嵌入式深度学习框架Tengine下利用深度学习算法搭建卷积神经网络,建立端到端可训练图像检测模型,并利用智能摄像机获取的图像数据训练模型;通过手眼标定获得摄像机坐标系与机械臂坐标系之间的关系,控制机械臂进行矸石追踪和分拣。实验结果表明,该系统矸石识别准确率稳定保持在95%以上,机械臂跟踪时间小于30 ms,执行误差为1 mm左右,可以满足煤矸分拣工艺要求。     

Development and field test of the articulated mobile robot T2 Snake-4 for plant disaster prevention

Abstract

In this work, we develop an articulated mobile robot that can move in narrow spaces, climb stairs, gather information, and operate valves for plant disaster prevention. The robot can adopt a tall position using a folding arm and gather information using sensors mounted on the arm. In addition, this paper presents a stair climbing method using a single backward wave. This method enables the robot to climb stairs that have a short tread. The developed robot system is tested in a field test at the World Robot Summit 2018, and the lessons learned in the field test are discussed.     

Application of a neural network to the generation of a robot arm trajectory

We propose a neural network model generating a robot arm trajectory. The developed neural network model is based on a recurrent-type neural network (RNN) model calculating the proper arm trajectory based on data acquired by evaluation functions of human operations as the training data. A self-learning function has been added to the RNN model. The proposed method is applied to a 2-DOF robot arm, and laboratory experiments were executed to show the effectiveness of the proposed method. Through experiments, it is verified that the proposed model can reproduce the arm trajectory generated by a human. Further, the trajectory of a robot arm is successfully modified to avoid collisions with obstacles by a self-learning function.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

基于M型标准块的视觉引导机器人系统手眼标定方法研究

针对线结构光传感器引导的机器人系统的手眼标定问题,提出了一种以M型标准块为标定物的方法。该M型标定物的两条平行的脊线作为约束,基于两条平行脊线的约束建立包含手眼关系、机器人运动学以及两条直线位姿参数误差的模型。首先基于定点约束求解手眼关系初值并以此为基础解算出直线位姿参数的初值,然后通过最小二乘法解算误差参数并补偿到模型中,不断迭代直至计算的误差参数小于阈值,最终得到最终的机器人手眼关系及运动学误差参数。为了验证标定方法的有效性,以某精加工平面为被测物,利用线结构光机器人系统对平面进行测量,得到平面点云;拟合最小二乘平面,计算点到平面距离的均方根值作为评价依据。分别对所述M型标准块和标准球两种方法进行了实验对比,结果表明,相较于标准球方法,所述M型标准块方法得到的均方根误差由0.152 mm减少到0.080 mm,均方根误差的标准差由0.043 mm减少到0.005 mm,其标定结果的精度及稳定性得到显著提高。     

Calibration of a Computer Controlled Robotic Vision Sensor with a Zoom Lens

Active vision sensors are increasingly being employed in vision systems for their greater flexibility. For example, vision sensors in hand-eye configurations with computer controllable lenses (e.g., zoom lenses) can be set to values which satisfy the sensing situation at hand. For such applications, it is essential to determine the mapping between the parameters that can actually be controlled in a reconfigurable vision system (e.g., the robot arm pose, the zoom setting of the lens) and the higher-level viewpoint parameters that must be set to desired values (e.g., the viewpoint location, focal length). In this paper we present calibration techniques to determine this mapping. In addition, we discuss how to use these relationships in order to achieve the desired values of the viewpoint parameters by setting the controllable parameters to the appropriate values. The sensor setup that is considered consists of a camera in a hand-eye arrangement equipped with a lens that has zoom, focus, and aperture control. The calibration techniques are applied to the H6 × 12.5R Fujinon zoom lens and the experimental results are shown.     

Grasp and Delivery for Moving Objects on Broken Lines

This paper studies the following variant of the Vehicle Routing Problem that we call the Grasp and Delivery for Moving Objects (GDMO) problem, motivated by robot navigation: The input to the problem consists of n products, each of which moves on a predefined path with a fixed constant speed, and a robot arm of capacity one. In each round, the robot arm grasps one product and then delivers it to the depot. The goal of the problem is to find a collection of tours such that the robot arm grasps and delivers as many products as possible. In this paper we prove the following results: (i) If the products move on broken lines with at least one bend, then the GDMO is MAXSNP-hard, and (ii) it can be approximated with ratio 2. However, (iii) if we impose the “straight line without bend” restriction on the motion of every product, then the GDMO becomes tractable. Y. Asahiro’s research was supported in part by Grant-in-Aid for Young Scientists 15700021, and Research on Priority Areas 16092223. E. Miyano’s research was supported in part by Grant-in-Aid for Research on Priority Areas 16092223. Presently, S. Shimoirisa is with the Software Development Engineering Department, Products Development Center, Retail Information Systems Company, Toshiba TEC Corporation.     

Development of a folding arm on an articulated mobile robot for plant disaster prevention

Abstract

In this work, we develop a folding arm on an articulated mobile robot to inspect an industrial plant. The design targets of the arm, its operations, measurement ability, and mobility, were set for the task of inspecting an industrial plant. To accomplish the targets, we designed the folding arm considering both accessibility to high locations and the mobility of the articulated mobile robot to which it is attached. The arm has links, joints, dummy wheels, and sensors and enables the robot to which it is attached to manipulate objects, e.g. rotating valves, opening a door, or inspecting by accessing high locations. In addition, changing the posture of the arm and touching the dummy wheel in the arm to the surrounding terrain can reduce any negative effect of the arm on the robot's mobility when it encounters narrow spaces, stairs, steps, and trenches. The arm is controlled as a six degrees-of-freedom manipulator without redundancy by an operator who directly sets two joint angles. The effectiveness of the developed arm was demonstrated not only through experiments in a laboratory but also in a field test at the Plant Disaster Prevention Challenge of the World Robot Summit 2018.     

基于手眼标定方程AX=XB的精度影响因素研究

机器人手位姿数据对手眼标定精度的影响不可忽略,将对基于手眼标定方程AX=XB的精度影响因素进行分析.通过手眼标定仿真和实测实验验证上述两个因素对手眼标定精度的影响与理论分析的一致性.通过仿真与实测实验,总结得出了减小摄像机与靶标间距离、减小机器人手的运动前后到基坐标空间距离的相差距离,可提高手眼标定精度,通过四元数法和矩阵直积法验证了此规律在解AX=XB标定方程时的通用性,并且在摄像机与靶标间距约为230 mm以及机器人手的运动前后到基坐标空间距离的相差距离为3.2401 mm时,手眼标定平移向量相对误差最高精度可达0.0403%.     

Supervisory control for co-ordination of multiple robots†

This paper deals with a solution of supervisory control problem for co-ordination of multiple robots. The application of two or more robots employed in a co-operating mode can increase the level of flexibility and productivity in an automated manufacturing environment. However, as the number of robots or the intricacy of task to be performed increases, the control requirements also increase substantially. Co-operating robots can be used in typically complex assembly operations where the payload may be too heavy for a single arm to handle; the object to be manipulated may be irregularly shaped and provide only limited access for physical handling, or the two parts to be assembled together may require simultaneous orientation and unique positioning.

The co-ordination control strategies developed were applied to two IBM 7540 SCARA robots. Several control schemes were devised to achieve concurrent robot co-ordination and trajectory tracking. A digital computer program operable on an IBM PS/2 System 80 provided multiple concurrent programming of the two robots. Interactive control strategy and master-slave control strategy were designed, implemented, optimized, and tested. The results showed excellent trajectory adherence and repeatability for both control schemes     

A real-time planning algorithm for obstacle avoidance of redundant robots

A computationally efficient, obstacle avoidance algorithm for redundant robots is presented in this paper. This algorithm incorporates the neural networks and pseudodistance function D _p in the framework of resolved motion rate control. Thus, it is well suited for real-time implementation. Robot arm kinematic control is carried out by the Hopfield network. The connection weights of the network can be determined from the current value of Jacobian matrix at each sampling time, and joint velocity commands can be generated from the outputs of the network. The obstacle avoidance task is achieved by formulating the performance criterion as D _p>d _min (d _min represents the minimal distance between the redundant robot and obstacles). Its calculation is only related to some vertices which are used to model the robot and obstacles, and the computational times are nearly linear in the total number of vertices. Several simulation cases for a four-link planar manipulator are given to prove that the proposed collision-free trajectory planning scheme is efficient and practical.     

一种用于机器人三维表面扫描系统的手眼标定算法

将线结构光测头安装在机器人末端,构成自由曲面非接触测量系统,利用齐次坐标变换理论建立测量系统的数学模型,针对该模型中的手眼标定问题,使用平面作为参考物,控制机器人以特定的位姿测量该平面,建立关于手眼关系的约束方程,并给出手眼参数的封闭解.通过对实际系统进行标定实验,验证了算法的有效性.     

Neural net robot controller: Structure and stability proofs

A multilayer neural net (NN) controller for a general serial-link robot arm is developed. The structure of the NN controller is derived using a filtered error approach. It is argued that standard backpropagation tuning, when used for real-time closed-loop control, can yield unbounded NN weights if: (1) the net can not exactly reconstruct a certain required control function, (2) there are bounded unknown disturbances in the robot dynamics, or (3) the robot arm has more than one link (i.e. nonlinear case). On-line weight tuning algorithms including correction terms to backpropagation, plus an added robustifying signal, guarantee tracking as well as bounded weights. The correction terms involve a second-orderforward-propagated wave in the backprop network.     

基于无迹卡尔曼滤波的机器人手眼标定

提出一种基于无迹卡尔曼滤波(UKF)的机器人在线手眼标定算法来求解齐次变换矩阵方程AX =XB.建立手眼标定的隐式马尔可夫模型(HMM),并对它进行无迹卡尔曼滤波,从而对标定参数的状态进行递归贝叶斯估计和实时可视化处理.蒙特卡洛仿真结果表明,在小高斯噪声、较大高斯噪声以及非等方向性高斯噪声模型下,本文算法估计结果的精确...