On the design of intelligent robotic agents for assembly

Robotic agents can greatly be benefited from the integration of perceptual learning in order to monitor and adapt to changing environments. To be effective in complex unstructured environments, robots have to perceive the environment and adapt accordingly. In this paper it is discussed a biology inspired approach based on the adaptive resonance theory (ART) and implemented on an KUKA KR15 industrial robot during real-world operations (e.g. assembly operations). The approach intends to embed naturally the skill learning capability during manufacturing operations (i.e., within a flexible manufacturing system).The integration of machine vision and force sensing has been useful to demonstrate the usefulness of the cognitive architecture to acquire knowledge and to effectively use it to improve its behaviour. Practical results are presented, showing that the robot is able to recognise a given component and to carry out the assembly. Adaptability is validated by using different component geometry during assemblies and also through skill learning which is shown by the robot’s dexterity.     

An anatomy of industrial robots and their controls

In the United States, commercially available industrial robots perform very well in limited areas of industrial tasks such as arc welding, paint spraying, etc. These tasks mainly involve synchronization but no task interaction. A close examination of the basic structure and controls of the robots reveals their resulting limitations which lead to unnatural specifications and inefficient performance of task interactions. It is our opinion that, to expand the range of robot tasks to include labor intensive jobs such as product assembly, sensors of multiple purposes must be added onto the robots and integrated into their control systems. Computer command language must be developed to enable nonexpert users to operate the robots, and a work-method must be available for analyzing robot time-motion so that the robots can be programmed to achieve best efficiency with least production cost.     

Design of service robots

According to the International Federation of Robotics (IFR), "a service robot is a robot which operates semi or fully autonomously to perform services useful to the well being of human and equipment, excluding manufacturing operations" [1]. These devices are typically complex systems requiring the input of knowledge from numerous disciplines. The authors have been using different software engineering techniques for the last 15 years, integrating new paradigms in the service robot development process as they emerged. This has made it possible to achieve rapid development of applications and subsequent maintenance. During the early years (1993?1998), our effortswere directed at the development of software for various kinds of teleoperated robots to performmaintenance tasks in nuclear power plants [2]; during a second phase (1999?2006),we built applications for ship-hull cleaning robots [3]. All this time, we have been applying all the possibilities of software engineering, from the use of paradigms for structured and object-based programming in early developments to the adoption of the current model-driven approach [model-driven engineering (MDE)] [4].     

Human–robot interaction in industrial collaborative robotics: a literature review of the decade 2008–2017

ABSTRACT

Currently, a large number of industrial robots have been deployed to replace or assist humans to perform various repetitive and dangerous manufacturing tasks. However, based on current technological capabilities, such robotics field is rapidly evolving so that humans are not only sharing the same workspace with robots, but also are using robots as useful assistants. Consequently, due to this new type of emerging robotic systems, industrial collaborative robots or cobots, human and robot co-workers have been able to work side-by-side as collaborators to accomplish tasks in industrial environments. Therefore, new human–robot interaction systems have been developed for such systems to be able to utilize the capabilities of both humans and robots. Accordingly, this article presents a literature review of major recent works on human–robot interactions in industrial collaborative robots, conducted during the last decade (between 2008 and 2017). Additionally, the article proposes a tentative classification of the content of these works into several categories and sub-categories. Finally, this paper addresses some challenges of industrial collaborative robotics and explores future research issues.     

Digital Twin for Human-Robot Interactive Welding and Welder Behavior Analysis

This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.     

Framework for Assessing Robotic Dexterity within Flexible Manufacturing

With growing demand for flexibility in manufacturing processes, interest in dexterous industrial robots is increasing. To facilitate benchmarking, and to assess the suitability of these robots for flexible manufacturing tasks, there is a need to develop new methods of capturing the relevant performance characteristics of industrial robots. This research aims to show that the Boothroyd-Dewhurst (B-D) Design-For-Assembly method, an established method for optimizing manufacturing processes, can be effectively adopted to form the basis of a comprehensive robotic dexterity assessment within flexible manufacturing. A comparative study is conducted which shows that the B-D classification tables offer the most comprehensive solution due to the range of operations and artifacts considered. Building on these tables, a framework is developed for determining the suitability of a robot system within flexible manufacturing operations. In a sample test-case scenario involving a pick-and-place operation, the framework is shown to produce an accurate estimate of robot performance that can be easily compared to human data. The framework establishes a link between manufacturing operations and robot performance metrics, which addresses the current difficulty in robot integration and highlights the framework’s potential for adoption within flexible manufacturing.     

Balancing task allocation in multi-robot systems using K-means clustering and auction based mechanisms

This paper aims to solve the balanced multi-robot task allocation problem. Multi-robot systems are becoming more and more significant in industrial, commercial and scientific applications. Effectively allocating tasks to multi-robots i.e. utilizing all robots in a cost effective manner becomes a tedious process. The current attempts made by the researchers concentrate only on minimizing the distance between the robots and the tasks, and not much importance is given to the balancing of work loads among robots. It is also found from the literature that the multi-robot system is analogous to Multiple Travelling Salesman Problem (MTSP). This paper attempts to develop mechanism to address the above two issues with objective of minimizing the distance travelled by ‘m’ robots and balancing the work load between ‘m’ robots equally. The proposed approach has two fold, first develops a mathematical model for balanced multi-robot task allocation problem, and secondly proposes a methodology to solve the model in three stages. Stage I groups the ‘N’ tasks into ‘n’ clusters of tasks using K-means clustering technique with the objective of minimizing the distance between the tasks, stage II calculates the travel cost of robot and clusters combination, stage III allocates the robot to the clusters in order to utilise all robot in a cost effective manner.     

Hard material small-batch industrial machining robot

Hard materials can be cost effectively machined with standard industrial robots by enhancing current state-of-the-art technologies. It is demonstrated that even hard metals with specific robotics-optimised novel hard-metal tools can be machined by standard industrial robots with an improved position-control approach and enhanced compliance-control functions. It also shows that the novel strategies to compensate for elastic robot errors, based on models and advanced control, as well as the utilisation of new affordable sensors and human-machine interfaces, can considerably improve the robot performance and applicability of robots in machining tasks. In conjunction with the development of safe robots for human-robot collaboration and cooperation, the results of this paper provide a solid background for establishing industrial robots for industrial-machining applications in both small- and medium-size enterprises and large industry. The planned short-term and long-term exploitation of the results should significantly increase the future robot usage in the machining operations.     

State-of-the-Art control strategies for robotic PiH assembly

Nowadays, industrial robots have been widely applied for performing position-controlled tasks with minimum contact such as spot welding, spray painting, packing, and material handling; however, performing high-tolerance assembly tasks still poses a great challenge for robots because of various uncertainties of the parts to be assembled such as fixtures, end effector tools, or axes. From this perspective, the advancement of research and development has led to cutting-edge robotic technologies for industrial applications. To understand the technological trend of industrial robots, investigated the state-of-the-art robotic assembly technologies to identify the limitations of existing works and clarify future research directions in the field. This paper especially interested on typical peg-in-hole (PiH) assemblies, as PiH methods provide insights for further development of robot assemblies. The assembly control strategies for PiH operations is classified by based on the types and features of the assemblies, and the literature in terms of the contributions of these studies is compared to PiH assembly. Finally, the control strategies for robotic PiH assemblies are discussed in detail, and the limitations of the current robotic assembly technologies are discussed to identify the future direction of research for the control of robotic assembly.     

Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach

We propose an approach to efficiently teach robots how to perform dynamic manipulation tasks in cooperation with a human partner. The approach utilises human sensorimotor learning ability where the human tutor controls the robot through a multi-modal interface to make it perform the desired task. During the tutoring, the robot simultaneously learns the action policy of the tutor and through time gains full autonomy. We demonstrate our approach by an experiment where we taught a robot how to perform a wood sawing task with a human partner using a two-person cross-cut saw. The challenge of this experiment is that it requires precise coordination of the robot’s motion and compliance according to the partner’s actions. To transfer the sawing skill from the tutor to the robot we used Locally Weighted Regression for trajectory generalisation, and adaptive oscillators for adaptation of the robot to the partner’s motion.     

Safe and intuitive manual guidance of a robot manipulator using adaptive admittance control towards robot agility

Key areas of robot agility include methods that increase capability and flexibility of industrial robots and facilitate robot re-tasking. Manual guidance can achieve robot agility effectively, provided that a safe and smooth interaction is guaranteed when the user exerts an external force on the end effector. We approach this by designing an adaptive admittance law that can adjust its parameters to modify the robot compliance in critical areas of the workspace, such as near and on configuration singularities, joint limits, and workspace limits, for a smooth and safe operation. Experimental validation was done with two tests: a constraint activation test and a 3D shape tracing task. In the first one, we validate the proper response to constraints and in the second one, we compare the proposed approach with different admittance parameter tuning strategies using a drawing task where the user is asked to guide the robot to trace a 3D profile with an accuracy or speed directive and evaluate performance considering path length error and execution time as metrics, and a questionnaire for user perception. Results show that appropriate response to individual and simultaneous activation of the aforementioned constraints for a safe and intuitive manual guidance interaction is achieved and that the proposed parameter tuning strategy has better performance in terms of accuracy, execution time, and subjective evaluation of users.     

Describing Upper-Body Motions Based on Labanotation for Learning-from-Observation Robots

We have been developing a paradigm that we call learning-from-observation for a robot to automatically acquire a robot program to conduct a series of operations, or for a robot to understand what to do, through observing humans performing the same operations. Since a simple mimicking method to repeat exact joint angles or exact end-effector trajectories does not work well because of the kinematic and dynamic differences between a human and a robot, the proposed method employs intermediate symbolic representations, tasks, for conceptually representing what-to-do through observation. These tasks are subsequently mapped to appropriate robot operations depending on the robot hardware. In the present work, task models for upper-body operations of humanoid robots are presented, which are designed on the basis of Labanotation. Given a series of human operations, we first analyze the upper-body motions and extract certain fixed poses from key frames. These key poses are translated into tasks represented by Labanotation symbols. Then, a robot performs the operations corresponding to those task models. Because tasks based on Labanotation are independent of robot hardware, different robots can share the same observation module, and only different task-mapping modules specific to robot hardware are required. The system was implemented and demonstrated that three different robots can automatically mimic human upper-body operations with a satisfactory level of resemblance.     

Collision-free and smooth joint motion planning for six-axis industrial robots by redundancy optimization

Planning collision-free and smooth joint motion is crucial in robotic applications, such as welding, milling, and laser cutting. Kinematic redundancy exists when a six-axis industrial robot performs five-dimensional tasks, and there are infinite joint configurations for a six-axis industrial robot to realize a cutter location data of the tool path. The robot joint motion can be optimized by taking advantage of the kinematic redundancy, and the collision-free joint motion with minimum joint movement is determined as the optimal. However, most existing redundancy optimization methods do not fully exploit the redundancy of the six-axis industrial robots when they conduct five-dimensional tasks. In this paper, we present an optimization method to solve the problem of inverse kinematics for a six-axis industrial robot to synthesize the joint motion that follows a given tool path, while achieving smoothness and collision-free manipulation. B-spline is applied for the joint configuration interpolation, and the sum of the squares of the first, second, and third derivatives of the B-spline curves are adopted as the smoothness indicators. Besides, the oriented bounding boxes are adopted to simplify the shape of the robot joints, robot links, spindle unit, and fixtures to facilitate collision detections. Dijkstra's shortest path technique and Differential Evolution algorithm are combined to find the optimal joint motion efficiently and avoid getting into a local optimal solution. The proposed algorithm is validated by simulations on two six-axis industrial robots conducting five-axis flank milling tasks respectively.     

Multiagent control of self-reconfigurable robots

We demonstrate how multiagent systems provide useful control techniques for modular self-reconfigurable (metamorphic) robots. Such robots consist of many modules that can move relative to each other, thereby changing the overall shape of the robot to suit different tasks. Multiagent control is particularly well-suited for tasks involving uncertain and changing environments. We illustrate this approach through simulation experiments of Proteo, a metamorphic robot system currently under development.     

Introduction of welding robots in shipyards

The great participation of direct human work characterizes today's shipbuilding industry. The actual status in the development of science and technology makes possible the replacement of humans with industrial robots in a large number of these work places. The strategy of the introduction of industrial robots in shipyards has to be adapted to existing working conditions, and the introduction has to be done gradually. This paper deals with a new method for priority setting of industrial robot work places and structures for welding operations in shipyards, based on the analytic hierarchy process. The numerical measure of priority of work places is based on the comparative pairwise judgments of social, psychological, technological, technical, safety, productivity and economical factors on different working locations. After the priority work places and priority working operations are chosen, the priority structures of adequate industrial robots are suggested according to their geometric, kinematic, dynamic and control characteristics.     

Physics-based simulation for manual robot guidance—An eRobotics approach

Manual robot guidance is an intuitive approach to teach robots with human's skills in the loop. It is particularly useful to manufacturers because of its high flexibility and low programming effort. However, manual robot guidance requires compliance control that is generally not available in position-controlled industrial robots. We address this issue from a simulation-driven approach. We systematically capture the interactive dynamic behavior of intelligent robot manipulators within physics-based virtual testbeds, regardless of the type of application. On this basis, we develop structures to equip and employ simulated robots with motion control capabilities that include soft physical interaction control driven in real-time with real external guidance forces. We then transfer the virtual compliant behavior of the simulated robots to their physical counterparts to enable manual guidance. The simulator provides assistance to operators through timely and insightful robot monitoring, as well as meaningful performance indexes. The testbed allows us to swiftly assess guidance within numerous interaction scenarios. Experimental case studies illustrate the practical usefulness of the symbiotic transition between 3D simulation and reality, as pursued by the eRobotics framework to address challenging issues in industrial automation.     

Book reviews

Abstract

Industrial robots often operate at high speed, with unpredictable motion patterns and erratic idle times. Serious injuries and deaths have occurred due to operator misperception of these robot design and performance characteristics. The main objective of the research project was to study human perceptual aspects of hazardous robotics workstations. Two laboratory experiments were designed to investigate workers' perceptions of two industrial robots with different physical configurations and performance capabilities. Twenty-four subjects participated in the study. All subjects were chosen from local industries, and had had considerable exposure to robots and other automated equipment in their working experience. Experiment 1 investigated the maximum speed of robot arm motions that workers, who were experienced with operation of industrial robots, judged to be ‘safe’ for monitoring tasks. It was found that the selection of safe speed depends on the size of the robot and the speed with which the robot begins its operation. Speeds of less than 51 cm/s and 63cm/s for large and small robots, respectively, were perceived as safe, i.e., ones that did not result in workers feeling uneasy or endangered when working in close proximity to the robot and monitoring its actions. Experiment 2 investigated the minimum value of robot idle time (inactivity) perceived by industrial workers as system malfunction, and an indication of the ‘safe-to-approach’ condition. It was found that idle times of 41 s and 28 s or less for the small and large robots, respectively, were perceived by workers to be a result of system malfunction. About 20% of the workers waited only 10 s or less before deciding that the robot had stopped because of system malfunction. The idle times were affected by the subjects' prior exposure to a simulated robot accident. Further interpretations of the results and suggestions for operational limitations of robot systems are discussed.     

A closed-loop approach for tracking a humanoid robot using particle filtering and depth data

Humanoid robots introduce instabilities during biped march that complicate the process of estimating their position and orientation along time. Tracking humanoid robots may be useful not only in typical applications such as navigation, but in tasks that require benchmarking the multiple processes that involve registering measures about the performance of the humanoid during walking. Small robots represent an additional challenge due to their size and mechanic limitations which may generate unstable swinging while walking. This paper presents a strategy for the active localization of a humanoid robot in environments that are monitored by external devices. The problem is faced using a particle filter method over depth images captured by an RGB-D sensor in order to effectively track the position and orientation of the robot during its march. The tracking stage is coupled with a locomotion system controlling the stepping of the robot toward a given oriented target. We present an integral communication framework between the tracking and the locomotion control of the robot based on the robot operating system, which is capable of achieving real-time locomotion tasks using a NAO humanoid robot.     

Solving a vehicle routing problem with resource conflicts and makespan objective with an application in car body manufacturing

We investigate the problem of dispatching arc welding robots in car body manufacturing. Such arc welding robots receive their energy from expensive laser sources. Laser sources can be shared among the robots. However, this requires that the robots be scheduled because each laser source can only be used by one robot at a time. We want to compute the minimal number of laser sources necessary to perform all welding tasks in a given processing time. To this end, we introduce the laser-sharing problem (LSP): for a given number of laser sources, find collision-free scheduled tours for all robots through all welding jobs so that the makespan is minimized. We propose a branch-and-bound algorithm for the LSP using bounds that stem from optimal solutions to carefully selected NP-hard combinatorial subproblems. This is the first algorithm for the LSP that is able to solve industrially relevant problem scales.     

An optimal path-generation algorithm for manufacturing of arbitrarily curved surfaces using uncalibrated vision

Multiple industrial manufacturing tasks require a complex path to be followed precisely over an arbitrary surface which has a geometry that is not known with precision. Examples of such tasks include welding, glue-application, cutting, plasma-spraying, etc., over commercial plates whose geometry cannot be known in advance. Such processes are in general referred to as surface manufacturing. In this work, a path is traced over the surface in an optimal fashion, using the concept of geodesic mapping. By definition, a geodesic line is the shortest line that joins two points over a surface whose algebraic representation is known. Such an optimal solution of a problem, associated with variational calculus, is the approach employed for mapping complex paths, defined in a data base, over a surface of arbitrary geometry. The straight-line segments in which a complex path can be divided are mapped onto an arbitrary surface as geodesic lines. The presented algorithm enables a user to interact with the system in a simple and efficient manner using a commercial computer pointing device. The algorithm was tested experimentally in an industrial maneuver involving arc welding, using an industrial robot and a method of vision-based, robot guidance known as camera-space manipulation. This method has the advantage of not requiring calibration of optical or mechanical components.