Team O2AS at the world robot summit 2018: an approach to robotic kitting and assembly tasks using general purpose grippers and tools

ABSTRACT

In this article, we propose a versatile robotic system for kitting and assembly tasks which uses no jigs or commercial tool changers. Instead of specialized end effectors, it uses its two-finger grippers to grasp and hold tools to perform subtasks such as screwing and suctioning. A third gripper is used as a precision picking and centering tool, and uses in-built passive compliance to compensate for small position errors and uncertainty. A novel grasp point detection for bin picking is described for the kitting task, using a single depth map. Using the proposed system we competed in the Assembly Challenge of the Industrial Robotics Category of the World Robot Challenge at the World Robot Summit 2018, obtaining 4th place and the SICE award for lean design and versatile tool use. We show the effectiveness of our approach through experiments performed during the competition.     

Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing

Quality and efficiency are crucial indicators of any manufacturing company. Many companies are suffering from a shortage of experienced workers across the production line to perform complex assembly tasks. To reduce time and error in an assembly task, a worker-centered system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The integrated AR is designed to provide on-site instructions including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is trained on a synthetic tool dataset. The dataset is generated using CAD models of tools and displayed onto a 2D scene without using real tool images. By experimenting the system to a mechanical assembly of a CNC carving machine, the result of a designed experiment shows that the system helps reduce the time and errors of the given assembly tasks by 33.2 % and 32.4 %, respectively. With the integrated system, an efficient, customizable smart AR instruction system capable of sensing, characterizing requirements, and enhancing worker’s performance has been built and demonstrated.     

Robots assembling machines: learning from the World Robot Summit 2018 Assembly Challenge

ABSTRACT

The Industrial Assembly Challenge at the World Robot Summit was held in 2018 to showcase the state-of-the-art of autonomous manufacturing systems. The challenge included various tasks, such as bin picking, kitting, and assembly of standard industrial parts into 2D and 3D assemblies. Some of the tasks were only revealed at the competition itself, representing the challenge of ‘level 5’ automation, i.e. programming and setting up an autonomous assembly system in less than one day. We conducted a survey among the teams that participated in the challenge and investigated aspects such as team composition, development costs, system setups as well as the teams' strategies and approaches. An analysis of the survey results reveals that the competitors have been in two camps: those constructing conventional robotic work cells with off-the-shelf tools, and teams who mostly relied on custom-made end effectors and novel software approaches in combination with collaborative robots. While both camps performed reasonably well, the winning team chose a middle ground in between, combining the efficiency of established play-back programming with the autonomy gained by CAD-based object detection and force control for assembly operations.     

Towards easy setup of robotic assembly tasks

ABSTRACT

There is a growing need for adaptive robotic assembly systems that are fast to setup and reprogram when new products are introduced. The World Robot Challenge at World Robot Summit 2018 was centered around the challenge of setting up a flexible robotic assembly system aiming at changeover times below 1 day. This paper presents a method for programming robotic assembly tasks that was initiated in connection with the World Robot Challenge that enables fast and easy setup of robotic insertion tasks.

We propose to program assembly tasks by demonstration, but instead of using the taught behavior directly, the demonstration is merged with assembly primitives to increase robustness. In contrast to other programming by demonstration approaches, we perform not only one demonstration but a sequence of four sub-demonstrations that are used to extract the desired robot trajectory in addition to parameters for the assembly primitive.

The proposed assembly strategy is compared to a standard dynamic movement primitive and experiments show that the proposed assembly strategy increases the robustness towards pose uncertainties and significantly reduces the applied forces during the execution of the assembly task.     

Learning Dextrous Manipulation Skills for Multifingered Robot Hands Using the Evolution Strategy

We present a method for autonomous learning of dextrous manipulation skills with multifingered robot hands. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning tractable. Our approach consists of learning and storing a few basic manipulation primitives for a few prototypical objects and then using an associative memory to obtain the required parameters for new objects and/or manipulations. The parameter space of the robot is searched using a modified version of the evolution strategy, which is robust to the noise normally present in real-world complex robotic tasks. Given the difficulty of modeling and simulating accurately the interactions of multiple fingers and an object, and to ensure that the learned skills are applicable in the real world, our system does not rely on simulation; all the experimentation is performed by a physical robot, in this case the 16-degree-of-freedom Utah/MIT hand. Experimental results show that accurate dextrous manipulation skills can be learned by the robot in a short period of time. We also show the application of the learned primitives to perform an assembly task and how the primitives generalize to objects that are different from those used during the learning phase.     

Learning Dextrous Manipulation Skills for Multifingered Robot Hands Using the Evolution Strategy

We present a method for autonomous learning of dextrous manipulation skills with multifingered robot hands. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning tractable. Our approach consists of learning and storing a few basic manipulation primitives for a few prototypical objects and then using an associative memory to obtain the required parameters for new objects and/or manipulations. The parameter space of the robot is searched using a modified version of the evolution strategy, which is robust to the noise normally present in real-world complex robotic tasks. Given the difficulty of modeling and simulating accurately the interactions of multiple fingers and an object, and to ensure that the learned skills are applicable in the real world, our system does not rely on simulation; all the experimentation is performed by a physical robot, in this case the 16-degree-of-freedom Utah/MIT hand. E xperimental results show that accurate dextrous manipulation skills can be learned by the robot in a short period of time. We also show the application of the learned primitives to perform an assembly task and how the primitives generalize to objects that are different from those used during the learning phase.     

Testing of a workstation efficiency evaluator tool

This paper presents three studies that test and characterize input errors and their impact on selected outputs of a method and tool to predict shoulder loading and work element time from seated, light assembly workstation layout. Studies examine the impact of 1) different hand loads on shoulder load prediction, 2) potential tool user hand location measurement errors, and 3) tool response to simulated hand location input error. The results of the first study found that predicted shoulder load in response to hand loads from newly created, load specific shoulder load models differed from the original method's base models by an average of 5%–12% within the first 100 g of hand loading and 150%–210% for 1000 g hand loads, depending on the population percentile being predicted. In the second study, average hand location measurement error of participants was within 1 cm of the desired referent value measured from either a virtual or physical workstation. Results in study three showed that the impact of simulated hand location measurement error on average shoulder load and movement time prediction was low (<0.05 Nm and <3.6 ms) and demonstrated the potential to cancel out individual task error over a number of work tasks. Combining the user input error results with the simulated error outputs demonstrated that the impact of human error is relatively low. Within parameters similar to these studies the tool should work well in the design-stage assessment of seated, light assembly workstations.Relevance to industryThis testing of a tool for predicting task time and shoulder loading from layout information in the design-stage of seated, light assembly workstations has shown acceptable performance for light assembly hand loads up to 100 g with an anticipated input error under 1 cm and low output errors for estimated shoulder load and movement time.     

Towards robot cell matrices for agile production – SDU Robotics' assembly cell at the WRC 2018

ABSTRACT

To support shifting to high mix/low volume production, manufacturers in high wage countries aim for robotizing their production operations – with a special focus on the late production phases, where robotic assembly cells are then confronted with any complexities resulting from part and product varieties. The ‘World Robot Challenge 2018’ (WRC 2018) emulated such high mix/low volume production scenarios in a competition taking place in Tokyo, Japan. As part of our activities in SDU's newly founded I4.0 Lab, we integrated and advanced our experiences and developments from our various R & D projects in a novel robotic assembly cell design to compete in the WRC 2018. This article describes the system architecture as well as main aspects of its implementation regarding robot control, robot programming and computer vision and how they contributed to winning the challenge. Due to the application of collaborative robots, the cell design allows for operation without fences. Hence, multiple copies of the cell can be arranged in a highly reconfigurable, highly adaptable matrix structure in which several production flows can be handled concurrently. This concept was demonstrated by the installation of a duplicate cell that allowed for parallel developments on two cells and prolonged development also after shipping the first cell to Japan.     

一种欠驱动水下机器人手爪的作业能力研究

在分析当前机器人手爪研究现状的基础上,根据水下作业任务特点,确定了作业型水下机器人多传感器手爪的设计需求,研究了水下手爪原理样机的作业能力,结合典型物体的抓取给出了关键参数关系.该样机可满足在指定环境下进行目标识别、抓取、简单装配、搬运等典型操作任务的要求,为操作型水下机器人多传感器手爪感知系统的研究提供了硬件平台.     

Autonomous industrial assembly using force,torque, and RGB-D sensing

ABSTRACT

We present algorithms and results for a robotic manipulation system that was designed to be easily programable and adaptable to various tasks common to industrial setting, which is inspired by the Industrial Assembly Challenge at the 2018 World Robotics Summit in Tokyo. This challenge included assembly of standard, commercially available industrial parts into 2D and 3D assemblies. We demonstrate three tasks that can be classified into ‘peg-in-hole’ and ‘hole-on-peg’ tasks and identify two canonical algorithms: spiral-based search and tilting insertion. Both algorithms use hand-coded thresholds in the force and torque domains to detect critical points in the assembly. After briefly summarizing the state of the art in research, we describe the strategy and approach utilized by the tested system, how it's design bears on its performance, statistics on 20 experimental trials for each task, lessons learned during the development of the system, and open research challenges that still remain.     

Plant inspection by using a ground vehicle and an aerial robot: lessons learned from plant disaster prevention challenge in world robot summit 2018

Abstract

In recent years, to prevent accidents and disaster are desired by implementing maintenance and management of facilities, such as conducting periodic inspections with appropriate frequency at plants. However, because the dangerous materials such as flammable gas and explosives is used in a plant, and there are many dangerous places in a plant such as high-temperature environment and high places and narrow spaces, it is desirable to use a remote-controlled robot for safety work and short inspections. Against this background, the Disaster Robotics Category-Plant Disaster Prevention Challenge was held in Japan at the World Robot Summit 2018. Our team was ranked 3rd in this competition, because our strategy of ‘inspection and investigation in cooperation with UGV and UAV’ was effective. In this paper, the competition contents of World Robot Summit 2018 and the robot inspection system that we are studying are explained. And what kind of strategy was challenged and result for these given competition tasks by using our robot system are introduced. And the lessons learned such as advantages and issues in UGV and UAV collaboration work at this competition are described for evaluate a robot investigation system for disaster response and inspection work at plants.     

Reasoning with contextual requirements: Detecting inconsistency and conflicts

ContextThe environment in which the system operates, its context, is variable. The autonomous ability of a software to adapt to context has to be planned since the requirements analysis stage as a strong mutual influence between requirements and context does exist. On the one hand, context is a main factor to decide whether to activate a requirement, the applicable alternatives to meet an activated requirement as well as their qualities. On the other hand, the system actions to reach requirements could cause changes in the context.ObjectivesModelling the relationship between requirements and context is a complex task and developing error-free models is hard to achieve without an automated support. The main objective of this paper is to develop a set of automated analysis mechanisms to support the requirements engineers to detect and analyze modelling errors in contextual requirements models.MethodWe study the analysis of the contextual goal model which is a requirements model that weaves together the variability of both context and requirements. Goal models are used during the early stages of software development and, thus, our analysis detects errors early in the development process. We develop two analysis mechanisms to detect two kinds of modelling errors. The first mechanism concerns the detection of inconsistent specification of contexts in a goal model. The second concerns the detection of conflicting context changes that arise as a consequence of the actions performed by the system to meet different requirements simultaneously. We support our analysis with a CASE tool and provide a systematic process that guides the construction and analysis of contextual goal models. We illustrate and evaluate our framework via a case study on a smart-home system for supporting the life of people having dementia problems.ResultsThe evaluation showed a significant ability of our analysis mechanisms to detect errors which were not notable by requirements engineers. Moreover, the evaluation showed acceptable performance of these mechanisms when processing up to medium-sized contextual goal models. The modelling constructs which we proposed as an input to enable the analysis were found easy to understand and capture.ConclusionsOur developed analysis for the detection of inconsistency and conflicts in contextual goal models is an essential step for the entire system correctness. It avoids us developing unusable and unwanted functionalities and functionalities which lead to conflicts when they operate together. Further research to improve our analysis to scale with large-sized models and to consider other kinds of errors is still needed.     

A mobile dual-arm manipulation robot system for stocking and disposing of items in a convenience store by using universal vacuum grippers for grasping items

ABSTRACT

A mobile dual-arm robot with universal vacuum grippers (UVG) for performing stocking and disposing tasks was developed. The robot grasps items in the tasks by using UVGs whose surface adapts to the various shapes of the items. A selection algorithm that determines whether the robot should use one or both manipulators to arrange an item was also developed. A ‘reachable-grasp pose’ is defined as a pose in which the robot’s UVG can easily place an item with a target attitude if it grasps the item. By using the selection algorithm, the robot re-grasps the item by adopting the reachable-grasp pose if the two manipulators do not collide when one is in the current grasp pose and the other is in the reachable-grasp pose. The robot system won the third prize of the Future Convenience Store Challenge 2018. In experiments on stocking and disposing tasks, the robot system achieved success rates of 100% for the stocking task and 80% for the disposing task. We believe that the results of this study will help researchers to develop a robot system for not only the stocking and disposing tasks but also other tasks in convenience stores (like customer interaction).     

Introducing autonomous aerial robots in industrial manufacturing

Although ground robots have been successfully used for many years in manufacturing, the capability of aerial robots to agilely navigate in the often sparse and static upper part of factories makes them suitable for performing tasks of interest in many industrial sectors. This paper presents the design, development, and validation of a fully autonomous aerial robotic system for manufacturing industries. It includes modules for accurate pose estimation without using a Global Navigation Satellite System (GNSS), autonomous navigation, radio-based localization, and obstacle avoidance, among others, providing a fully onboard solution capable of autonomously performing complex tasks in dynamic indoor environments in which all necessary sensors, electronics, and processing are on the robot. It was developed to fulfill two use cases relevant in many industries: light object logistics and missing tool search. The presented robotic system, functionalities, and use cases have been extensively validated with Technology Readiness Level 7 (TRL-7) in the Centro Bahía de Cádiz (CBC) Airbus D&S factory in fully working conditions.     

Task-board task for assembling a belt drive unit

ABSTRACT

A task board is a board for assembling parts, developed to test the abilities of a robot system to perform assembly operations associated with a manufacturing process. This study describes a task board that can serve as a basis for developing program modules for assembling a specific product. A properly designed task board can serve as a preliminary template for a specific assembly and can promote robotic assembly. This study analyzes the design of a Task-board task performed at the World Robot Summit 2018 (WRS 2018). The main competitive task at the WRS 2018 was the assembly of a belt drive unit. Analyses showed that the directions of assembly, especially the horizontal direction, were common difficulties in both Task-board and Assembly task competitions. Development of a task board for a specific assembly product requires an increase of its relevance to the assembly product, such as using the same workpieces or using similar assembly orientations.     

A compliant end-effector coupling for vertical assembly: design and evaluation

Many manipulation tasks require compliance, i.e. the robot's ability to comply with the environment and accomplish force as well as position control. Examples are constrained motion tasks and tasks associated with touch or feel in fine assembly. Few compliance-related tasks have been automated, and usually by active means of active compliance control: the need for passive compliance offered by the manipulator itself has been recognized and has led to the development of compliant end-effectors and/or wrists. In this paper we present a novel passively compliant coupling, the compliant end-effector coupling (CEEC), which aids automated precision assembly. It serves as a mechanical interface between the end of the robot arm and the end-effector. The coupling has 6 degrees of freedom. The design of the coupling is based on a “lock and free” assembly idea. The coupling is locked and behaves like a stiff member during robot motion, and is free (compliant) during constrained motion. It features an air bearing, a variable stiffness air spring and a center-locking mechanism. The end-effector assembly, being centrally unlocked, will float within the designed compliance limits assisted by the air bearing. These frictionless and constraint-free conditions facilitate a fast correction of any initial lateral and angular misalignments. In a peg insertion assembly, such accommodation is possible provided that the tip of the peg is contained within the chamfer of the hole. A variable stiffness air spring was incorporated in the design to allow variable and passive vertical compliance. This vertical compliance allows the accommodation of angular and vertical errors. The center-locking mechanism will return the end-effector assembly to its initial position upon an error correction. In a robot application program, the CEEC can be locked during rapid motion to securely transport a part or be set free during assembly or disassembly processes when the motions are constrained.     

Discrimination between design errors and user errors by binomial test

In laboratory usability studies, error collecting and analysis are frequently used methods of software evaluation and usually are capable of revealing the most conspicuous design deficiencies, if there are any. In certain cases, however, it is hard if not impossible to distinguish accidental user errors from systematic errors attributable to design failures without a careful statistical hypothesis testing. An appropriate, additional statistical analysis of the distribution of observed errors along the steps of the standard tasks could add further useful information to the obvious results of the study. For this purpose a simple binomial model is proposed, which proved to be applicable in a case study: by the help of this model it was possible to discover some further design failures in addition to the already obvious ones.     

The Postural Stability Diagram (PSD): personal constraints on the static exertion of force

Abstract

A chart called the Postural Stability Diagram (PSD) is used to represent static forces at the hands and feet during exertions. The constraints on exertion due to muscular strength and due to the distribution of body weight relative to the contacts of hands and feet with the workplace are discussed. Methods of collecting relevant anthropometric data are described, on which the personal statements about the capacity to exert forces are based. The PSD is intended for the analysis of tasks requiring static exertion. A following paper considers task-demands (environmental statements), which are superimposed on the personal statements on the chart during PSD analysis.     

Modeling and configuration-independent control of a self-mobile space manipulator

TheSelf-Mobile Space Manipulator (SM ²) is a 5-DOF, 1/3-scale, laboratory version of a robot designed to walk on the trusswork and other exterior surfaces of Space Station Freedom. It will be capable of routine tasks such as inspection, parts transportation and simple maintenance and repair. We have designed and built the robot and gravity compensation system to permit simulated zero-gravity experiments. The control ofSM ² is challenging because of significant structural flexibility, relatively high friction at the joints, positioning error amplified from joint errors due to the long reach, and high performance requirements for general 3-D locomotion. In this paper, we focus on the modeling of the robot system and the design of the control system based on the model. We address the kinematics and dynamic modeling of the 3-D motion ofSM ² and demonstrate the simulation and experimental modal analysis results. The robot dynamic characteristics vary significantly when the robot configuration changes. To consider this effect, we develop a control system that is composed of two basic parts, the model-based part and the servo part. The model-based loop can be updated based on the off-line dynamic model, and the servo control loop is updated by a gain schedule according to the off-line relationship between the closed-loop frequency and the modal frequency estimated from the off-line dynamic model. By taking dynamic variation into account in the controller, the control system is independent of the robot configuration, and the motion performance ofSM ² is greatly enhanced in implementation.     

Basic complexity criteria and their impact on manual assembly quality in actual production

Increasing design and assembly complexity are challenges facing the automotive industry today because increasing number of car variants and build options can result in immense difficulties and lead to costly assembly errors and quality losses. In order to remain on the market these conditions must nevertheless be managed by companies in hard competition with other manufacturers.The objective of this study was to analyze the impact of newly developed basic complexity criteria (CXB) on assembly quality and associated costs for corrective measures in manual assembly of cars. Data on error rate and action costs of assembly tasks of different complexity level was collected and analyzed. The inter-relationship between different complexity criteria was analyzed to see whether any criteria had a greater impact than others.The results showed that the action costs/car increased with increasing complexity level and that several complexity criteria together resulted in increased action costs. Some criteria tended to have a greater impact than others but need more research. The results further suggest that if high complexity issues are identified and replaced by low complexity solutions the assembly related action costs in manual assembly are likely to decrease.Relevance to industryBy reduction of basic assembly complexity already in early planning stages in product development significant reduction of costly assembly related action costs in manual assembly can probably be made.