Vision-guided fixtureless assembly of automotive components

Assembly operations in many industries make extensive use of fixtures that are costly and inflexible. The goal of “robotic fixtureless assembly” (RFA) is to replace these fixtures with sensor-guided robots. In this paper, the development of a vision-guided RFA workcell for automotive components is described. Each robot is equipped with a multiple degree-of-freedom programmable gripper, allowing it to hold a wide range of part shapes without tool changing. A 2D computer vision is used to achieve part pickup which is robust to positioning errors. A novel 3D computer vision system is used to align the parts prior to joining them. The actions of the workcell devices are coordinated using a flexible distributed object-oriented approach. Experimental results are presented for the RFA of four automotive body components.     

Two-arm kinematic posture optimization for fixtureless assembly

This article focuses on finding optimal kinematic postures for two robot arms performing various assembly tasks without the aid of jigs and fixtures. The goal of the work is to identify the manipulator postures that would result in some optimum kinematic conditions for initiating a typical male-female assembly. Specifically, the article addresses issues related to the following question: In the absence of preset jigs and fixtures, where are the optimum location in the 3-dimensional work space and the optimum orientations of the assembly parts (therefore, the postures of the manipulators holding the parts) to perform a fixtureless assembly? First, using velocity and force ellipsoids a performance criterion is developed for the optimal kinematic posture based on constraints that are imposed by the kinematic coupling of the two robot arms as well as the environment. Next, an algorithm based on a null-space search technique is formulated to “search” for the posture of the two-arm system that maximizes the performance criterion within the set of postures that satisfy given constraints. Finally, the algorithm is demonstrated numerically for two planar robots performing typical fixtureless assembly operations. © 1995 John Wiley & Sons, Inc.     

Three‐dimensional localization of thin‐walled sheet metal parts for robotic assembly

This article presents a technical demonstration of a system for determining the three‐dimensional spatial location of complexly shaped, thin‐walled sheet metal parts grasped by robots during assembly. For successful part assembly, the precise location of grasped parts (essential for successful mating of parts) must be achieved. A localization system is implemented to determine the accurate position and orientation of a sheet metal part that has been picked up by a robot from an arbitrary location. The proposed localization system employs a novel sensing method, utilizing laser‐based proximity and edge detectors, to extract the part feature data in real time. These geometrical feature data are incorporated into an existing localization algorithm, which is based on the singular value decomposition formulation of the part localization problem. The sensing method is particularly effective in measuring 3‐D feature geometry (i.e., thin edges) of sheet metal parts. An experimental single‐robot test bed has been developed to demonstrate the feasibility of the part localization concept for a single sheet metal part. The experimental results obtained from the test bed demonstrate that the system can be effectively used for the localization of thin‐walled sheet metal parts. © 2002 Wiley Periodicals, Inc.     

Application of robots in assembly automation

The development of flexible assembly is closely related to the introduction of robots in assembly automation. If has long been recognized that automatic parts assembly by robots is one of the most delicate and most difficult tasks in industrial robotics. This task involves two control problems, trajectory planning for the whole automatic assembly process and reduction of the reaction forces appearing between the parts being assembled. This paper addresses both aspects of this control task. The strategical control level for the manipulation of robots and various approaches to trajectory planning tasks in assembly processes are discussed. A new approach to the determination of the strategical control level, including various models (geometric, kinematic and dynamic) for manipulation robots, is briefly described.The last and most delicate phase of the assembly process is parts mating, which is rather like inserting a peg in a hole. In order to reduce the reaction forces appearing between the parts being assembled, force feedback control is applied. The experimental results of the industrial robot insertion process with force feedback are also presented in the paper.     

Computer-aided analysis of reconfigurable fixtures and sheet metal parts for robotic drilling

This paper presents an automated manufacturing system for drilling sheet metal parts. All stand-alone systems such as the robot, a set of reconfigurable fixtures and the CAD/CAM workstation have been integrated into a flexible manufacturing system. This system analyzes and evaluates a given fixturing layout and assembles the reconfigurable fixtures automatically using a robot manipulator. An optimum fixturing layout and assembly are achieved by examining the workpart from a stress-strain point of view. In addition, issues such as geometric constraints, yielding and buckling, database design, collision detection, fixturing sequential control and the automatic assembly of fixture elements are considered. The computational and analytical concepts for the reconfigurable fixturing and drilling system are also presented in this paper.     

Analysis of autonomous cooperative assembly using coordination schemes by heterogeneous robots using a control basis approach

Robotic technology is quickly evolving allowing robots to perform more complex tasks in less structured environments with more flexibility and autonomy. Heterogeneous multi-robot teams are more common as the specialized abilities of individual robots are used in concert to achieve tasks more effectively and efficiently. An important area of research is the use of robot teams to perform modular assemblies. To this end, this paper analyzed the relative performance of two robots with different morphologies and attributes in performing an assembly task autonomously under different coordination schemes using force sensing through a control basis approach. A rigid, point-to-point manipulator and a dual-armed pneumatically actuated humanoid robot performed the assembly of parts under a traditional “push-hold” coordination scheme and a human-mimicked “push-push” scheme. The study revealed that the scheme with higher level of cooperation—the “push-push” scheme—performed assemblies faster and more reliably, lowering the likelihood of stiction phenomena, jamming, and wedging. The study also revealed that in “push-hold” schemes industrial robots are better pushers and compliant robots are better holders. The results of our study affirm the use of heterogeneous robots to perform hard-to-do assemblies and also encourage humans to function as holder’s when working in concert with a robot assistant for insertion tasks.     

State-of-the-Art control strategies for robotic PiH assembly

Nowadays, industrial robots have been widely applied for performing position-controlled tasks with minimum contact such as spot welding, spray painting, packing, and material handling; however, performing high-tolerance assembly tasks still poses a great challenge for robots because of various uncertainties of the parts to be assembled such as fixtures, end effector tools, or axes. From this perspective, the advancement of research and development has led to cutting-edge robotic technologies for industrial applications. To understand the technological trend of industrial robots, investigated the state-of-the-art robotic assembly technologies to identify the limitations of existing works and clarify future research directions in the field. This paper especially interested on typical peg-in-hole (PiH) assemblies, as PiH methods provide insights for further development of robot assemblies. The assembly control strategies for PiH operations is classified by based on the types and features of the assemblies, and the literature in terms of the contributions of these studies is compared to PiH assembly. Finally, the control strategies for robotic PiH assemblies are discussed in detail, and the limitations of the current robotic assembly technologies are discussed to identify the future direction of research for the control of robotic assembly.     

Robotic assembly automation using robust compliant control

Industrial robots used to perform assembly applications are still a small portion of total robot sales each year. One of the main reasons is that it is difficult for conventional industrial robots to adapt to any sort of change. This paper proposes a robust control strategy to perform an assembly task of inserting a printed circuit board (PCB) into an edge connector socket using a SCARA robot. The task is very challenging because it involves compliant manipulation in which a substantial force is needed to accomplish the insertion operation and there are some dynamic constraints from the environment. Therefore, a robust control algorithm is developed and used to perform the assembly process. The dynamic model of the robotic system is developed and the dynamic parameters are identified. Experiments were performed to validate the proposed method. Experimental results show that the robust control algorithm can deal with parameter uncertainties in the dynamic model, thus achieve better performance than the model based control method. An abnormal case is also investigated to demonstrate that the robust compliant control method can deal with the abnormal situation without damaging the system and assembly parts, while pure position control method may cause damages. This strategy can also be used in other similar assembly processes with compliant applications.     

Hierarchical multi-agent planning for flexible assembly of large-scale lunar facilities

Several national space agencies and commercial aerospace companies plan to set up lunar bases with large-scale facilities that rely on multiple lunar robots’ assembly. Mission planning is necessary to achieve efficient multi-robot cooperation. This paper aims at autonomous multi-robot planning for the flexible assembly of the large-scale lunar facility, considering the harsh lunar environment, mission time optimization, and joint actions. The lunar robots and modules are scattered around the mission area without fixed assembly lines. Thus, the traditional assembly planning methods ignoring the optimal selection of modules are unable to handle this problem. We propose a hierarchical multi-agent planning method based on two-stage two-sided matching (HMAP-TTM) to solve this critical problem. First, the distributed planning framework with multi-replica public agents is introduced, ensuring robot plan knowledge consistency through public agents’ communication. Second, the hierarchical task graph (HTG) divides the mission into task layers based on task dependency knowledge. Third, we develop a novel two-stage two-sided matching algorithm. Time-optimal plans emerge from the matching games among public and private agents in each layer of HTG. Agents make decisions in the game based on action knowledge updated during planning. Finally, an assembly mission is presented to prove the method’s effectiveness. The simulation results show that the HMAP-TTM can generate plans with shorter mission time and require smaller communication costs than the baseline methods.     

Artificial intelligence and robotic assembly

Traditionally, most industrial robots are programmed by teaching. The emergence of robot-level programming languages has improved the programmer's ability to describe and modify the robot moves. However, commercially available robot-level programming languages still fall short of the robot user's need to program complex tasks, and consequently, are not widely used in industry. There is an increasing need for integrating sensors feedback into the robot system to provide better perception and for improving the capacity of the robot to reason and make decisions intelligently in real time.The role of artificial intelligence in programming and controlling robots is discussed. Available robot programming systems including robot-level, object-level, and task-level languages are reviewed. The importance of developing intelligent robots in broadening the scope of flexible automation and opening the door to new robotic applications in space, under water and in harsh environments is outlined. The current development and implementation of programming and control systems for intelligent robots, at McMaster University, are explained. A number of research issues are discussed such as (1) automatic task planning, (2) knowledge representation and use, (3) world modeling, (4) reasoning in automatic assembly planning, and (5) vision monitoring of actions. Examples of geometric, functional, and handling reasoning, as they apply to assembly, are provided. The systems described in this paper are being implemented in the center for flexible manufacturing research and development. Several pieces of hardware are used, including a six-axis articulated robot, a grey-level vision system with a multi-camera, Micro VAX II, and a variety of graphics monitors. The languages available for software development include Common LISP, C, OPS5, VAL II, PASCAL, and FORTRAN 77. The domain of application is currently focused on mechanical assembly.     

Using simulation to test the feasibility of robotic assembly

In this paper we address the use of simulation to test the feasibility of robotic assembly. It is shown that three factors are important in modeling an assembly problem: the geometry of the mating parts, the tolerance of assembly components, and the repeatability of the robot system. The process of developing a simulation model is demonstrated using two generic classes of assembly geometries. Simulation programs are presented and the implications for computer-aided design are discussed.     

Design specification of parts dimensional tolerance for robotic assembly

Success of robotic assembly can be limited by parts dimensional tolerances as well as robot repeatability. This paper examines the effects of these limitations on the success (or failure) of parts assembly. A Positioning Design Factor for Assembly (pdfa) is defined to represent the percentage of all cases where the combination of parts dimensional tolerances and parts positioning errors result in positive effective clearance, which is one of the conditions leading to successful assembly. Mathematical derivations have been applied to the classical peg-in-hole problem. The dimensional tolerances of the peg and hole as well as their relative position are considered to be randomly distributed variables. Robot repeatability was also modelled as a random variable within the robot specification. A Monte Carlo analysis was used to predict the probability of favourable positioning conditions for successful insertions. The simulation also incorporated several common assembly situations such as the effect of chamfers on the mating parts and the ansi Standard fit classes.The results have been condensed in this paper into a family of charts which relate the Positioning Design Factor for Assembly to parts size and different parts fit classes. A design chart which gives maximum allowable positioning error for the best pdfa versus parts size is presented. This chart can assist the designer in specifying and selecting both robots repeatability limits and parts dimensional tolerances for assembly applications.     

Tolerance dataset: mating process of plug-in cable connectors for wire harness assembly tasks

In this research, we studied the mating tolerance of various plug-in cable connectors and provide a mating tolerance dataset of 70 different connectors. This dataset will be highly advantageous to industries for wire harness assembly tasks using robots. Understanding the mating tolerance is crucial for automating the mating process because it is closely related to the control specifications of a robotic manipulator. Our system uses a 2-finger Robotiq adaptive gripper attached to a 6 degree-of-freedom industrial robot (ABB Robotics) to test the mating process of wire harness assembly tasks. In addition, we use 70 types of wire harness connectors with different numbers of pins widths, lengths, and thicknesses, and various shapes, to test the mating tolerance. The results indicate that the connector mating tolerance of our dataset is more generous than the repeatability of conventional industrial manipulators, and further demonstrate the suitability of the position control methods to wire harness assembly tasks.

     

Knowledge model-based heterogeneous multi-robot system implemented by a software platform

In order to build a heterogeneous multi-robot system that can be regarded as a primitive prototype of a future symbiotic autonomous human-robot system, this paper presents a knowledge model-based heterogeneous multi-robot system implemented by a software platform. With using frame-based knowledge representation, a knowledge model is constructed to describe the features of heterogeneous robots as well as their behaviors according to human requests. The required knowledge for constructing a heterogeneous multi-robot system can be therefore integrated in a single model and shared by robots and users. Based on the knowledge model, the heterogeneous multi-robot system is defined in the Software Platform for Agents and Knowledge Management (SPAK) by use of XML format. With the use of SPAK, the cooperative operation of heterogeneous multi-robot system can be flexibly carried out. The proposed system not only can integrate heterogeneous robots and various techniques for robots, but also can automatically perform human-robot interaction and plan robot behaviors taking into account different intelligence of robots corresponding to human requests. In this paper, an actual heterogeneous multi-robot system comprised by humanoid robots (Robovie, PINO), mobile robot (Scout) and entertainment robot dog (AIBO) is built and the effectiveness of the proposed system is verified by experiment.     

断路器柔性装配车间数字孪生系统设计

为优化断路器装配车间的产线结构和作业方法,结合数字孪生技术,提出一种基于多机器人运动控制的断路器柔性自动化车间装配方案。面向自动化装配单元,结合工业机器人的柔性装配工艺及方法,对实体装配车间进行全物理属性的数字化建模,同时建立多机器人的运动学控制模型,将机器人的三维运动模型应用于虚拟孪生场景。通过数据的交互传递,实现物理单元与虚拟单元的实时链接,将车间机器人的运动轨迹、装配状态、作业运送流程等数据信息进行实时显示,从而实现断路器柔性装配数字孪生系统的搭建与同步映射。实验结果证明,所提方案对实现断路器柔性装配有显著效果。     

A review on vision-based control of flexible manipulators

In this review, recent developments in the field of flexible robot arm control using visual servoing are reviewed. In comparison to rigid robots, the end-effector position of flexible links cannot be obtained precisely enough with respect to position control using kinematic information and joint variables. To solve the task here the use of a vision sensor (camera) system, visual servoing is proposed to realize the task of control flexible manipulators with improved quality requirements. The paper is organized as follows: the visual servoing architectures will be reviewed for rigid robots first. The advantages, disadvantages, and comparisons between different approaches of visual servoing are carried out. The using of visual servoing to control flexible robot is addressed next. Open problems such as state variables estimation as well as the combination of different sensor properties as well as some application-oriented points related to flexible robot are discussed in detail.     

Robot control strategy for in-orbit assembly of a micro satellite

We have developed novel heuristic control strategies for fitting parts with almost no clearance and also dealing with flexible objects using visual and force feedback. We have applied these control strategies for micro satellite assembly tasks of a ground research model of an in-orbit maintenance robot system. For the in-orbit maintenance system, micro satellites should be modularized as much as possible. The in-orbit maintenance system needs to assemble the modularized parts into micro satellites. We have developed a ground research model including a two-armed robotic platform and modularized micro satellite models. To assemble a micro satellite model, two problems arise: one is to engage two parts into a tight fit and the other is to handle flexible parts. We use a heuristic approach to solve the first problem — to grope one part to find the entrance of engagement of the other and to wobble this part to make a tight fit. For the second problem, visual measurement of the parts is used to position the end-effector of a robot arm and also active limp control is extensively used to adjust any misalignment that arises from the visual measurement error. With the combination of the heuristic, visual and active limp control, the system can successfully assemble a micro satellite.     

Robust multi-robot coordination in pick-and-place tasks based on part-dispatching rules

This paper addresses the problem of realizing multi-robot coordination that is robust against pattern variation in a pick-and-place task. To improve productivity and reduce the number of parts remaining on the conveyor, a robust and appropriate part flow and multi-robot coordinate strategy are needed. We therefore propose combining part-dispatching rules to coordinate robots, by integrating a greedy randomized adaptive search procedure (GRASP) and a Monte Carlo strategy (MCS). GRASP is used to search for the appropriate combination of part-dispatching rules, and MCS is used to estimate the minimum-maximal part flow for one combination of part-dispatching rules. The part-dispatching rule of first-in–first-out is used to control the final robot in the multi-robot system to pick up parts left by other robots, and the part-dispatching rule of shortest processing time is used to make the other robots pick up as many parts as possible. By comparing it with non-cooperative game theory, we verify that the appropriate combination of part-dispatching rules is effective in improving the productivity of a multi-robot system. By comparing it with a genetic algorithm, we also verify that MCS is effective in estimating minimum-maximal part flow. The task-completion success rate derived via the proposed method reached 99.4% for 10,000 patterns.     

A model for output evaluation of an intelligent robotic assembly station

The evolution of intelligent robots has brought widespread changes in the concept of automatic assembly. An intelligent robot can successfully accomplish assembly in a multiple end product processing and unstructured component part(s) handling environment. This advanced capability of the robot dictates a major change in their basic output characteristics. For example, task service time of an intelligent assembly robot is no longer completely deterministic like that of an ordinary pick-and-place type robot. In this paper, a typical component insertion type assembly task is analyzed to show that the task service time is a random variable consisting of two (2) parts, a deterministic component and a stochastic component. Thus, with the assumption of exponential part input an assembly station served by a single intelligent robot is modeled as an M/G/1 queueing model. A robot availability factor is also introduced into the service time random variable. Closed form expressions for important system performance parameters are obtained under the assumption of three (3) different probability distributions for the stochastic part of the service time random variable.     

Cooperative Control of a 3D Dual-Flexible-Arm Robot

This paper discusses cooperative control of a dual-flexible-arm robot to handle a rigid object in three-dimensional space. The proposed control scheme integrates hybrid position/force control and vibration suppression control. To derive the control scheme, kinematics and dynamics of the robot when it forms a closed kinematic chain is discussed. Kinematics is described using workspace force, velocity and position vectors, and hybrid position/force control is extended from that on dual-rigid-arm robots. Dynamics is derived from constraint conditions and the lumped-mass-spring model of the flexible robots and an object. The vibration suppression control is calculated from the deflections of the flexible links and the dynamics. Experiments on cooperative control are performed. The absolute positions/orientations and internal forces/moments are controlled using the robot, each arm of which has two flexible links, seven joints and a force/torque sensor. The results illustrate that the robot handled the rigid object damping links' vibration successfully in three-dimensional space.