Augmented reality-assisted robot programming system for industrial applications

Robots are important in high-mix low-volume manufacturing because of their versatility and repeatability in performing manufacturing tasks. However, robots have not been widely used due to cumbersome programming effort and lack of operator skill. One significant factor prohibiting the widespread application of robots by small and medium enterprises (SMEs) is the high cost and necessary skill of programming and re-programming robots to perform diverse tasks. This paper discusses an Augmented Reality (AR) assisted robot programming system (ARRPS) that provides faster and more intuitive robot programming than conventional techniques. ARRPS is designed to allow users with little robot programming knowledge to program tasks for a serial robot. The system transforms the work cell of a serial industrial robot into an AR environment. With an AR user interface and a handheld pointer for interaction, users are free to move around the work cell to define 3D points and paths for the real robot to follow. Sensor data and algorithms are used for robot motion planning, collision detection and plan validation. The proposed approach enables fast and intuitive robotic path and task programming, and allows users to focus only on the definition of tasks. The implementation of this AR-assisted robot system is presented, and specific methods to enhance the performance of the users in carrying out robot programming using this system are highlighted.     

Artificial intelligence and robotic assembly

Traditionally, most industrial robots are programmed by teaching. The emergence of robot-level programming languages has improved the programmer's ability to describe and modify the robot moves. However, commercially available robot-level programming languages still fall short of the robot user's need to program complex tasks, and consequently, are not widely used in industry. There is an increasing need for integrating sensors feedback into the robot system to provide better perception and for improving the capacity of the robot to reason and make decisions intelligently in real time.The role of artificial intelligence in programming and controlling robots is discussed. Available robot programming systems including robot-level, object-level, and task-level languages are reviewed. The importance of developing intelligent robots in broadening the scope of flexible automation and opening the door to new robotic applications in space, under water and in harsh environments is outlined. The current development and implementation of programming and control systems for intelligent robots, at McMaster University, are explained. A number of research issues are discussed such as (1) automatic task planning, (2) knowledge representation and use, (3) world modeling, (4) reasoning in automatic assembly planning, and (5) vision monitoring of actions. Examples of geometric, functional, and handling reasoning, as they apply to assembly, are provided. The systems described in this paper are being implemented in the center for flexible manufacturing research and development. Several pieces of hardware are used, including a six-axis articulated robot, a grey-level vision system with a multi-camera, Micro VAX II, and a variety of graphics monitors. The languages available for software development include Common LISP, C, OPS5, VAL II, PASCAL, and FORTRAN 77. The domain of application is currently focused on mechanical assembly.     

Adaptive robot training by programming and guiding

The simplest and most common robot training method is operator-guiding, but this method is inflexible and limited to fixed sequences. Explicit textual robot programming has enjoyed many advances in the last decade and current research implementations are powerful. But explicit programming is complex and programmers must be specially trained. Mixed systems attempt to unite the facility of guiding with the power of programming. However, existing mixed systems exhibit mismatches in the interaction between programming and guiding, because they ignore underlying dependencies between the programmer and operator. An operator may (1) be given insufficient visual cues; (2) lack the required dexterity; or (3) be required to add unforeseen movement sequences to the program. This paper presents the design and implementation of ART (adaptive robot trainer), a prototypical mixed robot training system that eliminates or corrects deficiencies in visual cues and dexterity, and additionally improves the guiding and programming components. Mixed systems and assembly tasks are analysed, to give an effective representation of task state, which in turn motivates the design of ART's language to automate much of the program-guiding interaction. The language allows the programmer to express assembly operations and object-feature relationships in a natural way while providing the system with the information necessary to maintain the task state. The representation also enables the correction of guiding errors, flexibility in the guiding protocol and the generation of meaningful messages to prompt operator actions. These principles in the design and implementation pave the road to more instructable, capable robots.Now at Bell Northern Research, Ottawa.     

Automatic construction of systems of distributed components from nested Petri nets models

Multi-level multi-agent systems (MASs) with dynamic structure are widely used in solving important applied problems in telecommunication, transportation, social, and other systems. Therefore, ensuring correct behavior of such systems is an actual and important task. One of the most error-prone stages of system development in the framework of model-oriented approach is the implementation stage, in the course of which a program code is constructed based on the model developed. This paper presents an algorithm for automated translation of MAS models represented as nested Petri nets into systems of distributed components. Nested Petri nets are the extension of Petri nets in the framework of the nets-within-nets approach, which assumes that tokens in a Petri net may themselves be Petri nets, possess autonomous behavior, and interact with other tokens of the net. This makes it possible to model MASs with dynamic structure in a natural way. The translation presented in this paper preserves distribution level and important behavioral properties (safety, liveness, and conditional liveness) of the original model and ensures fairness of the target system execution. The use of such translation makes it possible to automate construction of distributed MASs by models of nested Petri nets. As a test example, translation of nested Petri nets into systems of distributed components was implemented on the basis of the EJB component technology.     

Coloured Petri nets and graphical simulation for the validation of a robotic cell in aircraft industry

This paper proposes a mixed validation approach based on coloured Petri nets and 3D graphic simulation for the design of supervisory systems in manufacturing cells with multiple robots. The coloured Petri net is used to model the cell behaviour at a high level of abstraction. It models the activities of each cell component and its coordination by a supervisory system. The graphical simulation is used to analyse and validate the cell behaviour in a 3D environment, allowing the detection of collisions and the calculation of process times. The motivation for this work comes from the aeronautic industry. The automation of a fuselage assembly process requires the integration of robots with other cell components such as metrological or vision systems. In this cell, the robot trajectories are defined by the supervisory system and results from the coordination of the cell components. The paper presents the application of the approach for an aircraft assembly cell under integration in Brazil. This case study shows the feasibility of the approach and supports the discussion of its main advantages and limits.     

运输机器人行为建模的Petri网方法

针对建立运输机器人的精准行为模型所遇到的新困难,提出采用含禁止弧Petri网（PN）建立其行为模型。运输机器人行为有耦合、制约、异步等特点,采用含禁止弧的行为交互Petri网元模型以及token流动控制机制建模其行为。通过LabVIEW2012及Robotics模块,将Petri网模型转化为LabVIEW程序,在运输机器人平台进行行为验证。结果实现了运输机器人的行为与交互逻辑,具有行为辨识、决策与执行能力。验证了含禁止弧Petri网为运输机器人的行为建模提供了一种适用方法,建立的Petri网模型为运输机器人的相关行为设计提供模型参考。     

Constraint-based modeling of discrete event dynamic systems

Numerous frameworks dedicated to the modeling of discrete event dynamic systems have been proposed to deal with programming, simulation, validation, situation tracking, or decision tasks: automata, Petri nets, Markov chains, synchronous languages, temporal logics, event and situation calculi, STRIPS…All these frameworks present significant similarities, but none offers the flexibility of more generic frameworks such as logic or constraints. In this article, we propose a generic constraint-based framework for the modeling of discrete event dynamic systems, whose basic components are state, event, and time attributes, as well as constraints on these attributes, and which we refer to as CNT for Constraint Network on Timelines. The main strength of such a framework is that it allows any kind of constraint to be defined on state, event, and time attributes. Moreover, its great flexibility allows it to subsume existing apparently different frameworks such as automata, timed automata, Petri nets, and classical frameworks used in planning and scheduling.     

Petri网在自动焊接机机械手程序设计中的应用

可编程控制器(PLC)在自动化生产过程中有着广泛的应用,它的程序设计主要通过梯形图实现。本文阐述了采用Petri网进行的程序设计和分析,能够更加准确描述PLC的控制过程,并重点介绍了Petri网替代梯形图进行自动焊接机机械手程序设计的方法和步骤。     

A simplification of a conversation design scheme using Petri nets

In the conversation design procedure, the definition of the state of the system is one of the most important aspects. The question is how to identify transitions in Occam programs in order to express them as Petri nets. In the paper, a simplified transition identification method is proposed. Using the robot arm control program of A.M. Tyrrell and D.J. Holding it is shown that the correspondent Petri net graph is simpler than theirs, but the communication state change table is the same. It is also shown that these two methods are equivalent     

Steady-state performance evaluation of continuous mono-T-semiflow Petri nets

The number of states in discrete event systems can increase exponentially with respect to the size of the system. A way to face this state explosion problem consists of relaxing the system model, for example by converting it to a continuous one. In the scope of Petri nets, the firing of a transition in a continuous Petri net system is done in a real amount. Hence, the marking (state) of the net system becomes a vector of non-negative real numbers. The main contribution of the paper lies in the computation of throughput bounds for continuous Petri net systems with a single T-semiflow. For that purpose, a branch and bound algorithm is designed. Moreover, it can be relaxed and converted into a linear programming problem. Some conditions, under which the system always reaches the computed bounds, are extracted. The results related to the computation of the bounds can be directly applied to a larger class of nets called mono T-semiflow reducible.     

基于平行Petri网的制造系统调度与控制一体化方法

为了消除制造系统调度层与控制层之间的隔阂,实现对生产事件快速灵活响应,本文提出了一种调度与控制一体化的方法.首先,定义了一种新型Petri网模型,即平行Petri网,从而集成地描述了传感器、执行器、任务和资源信息,构建制造系统的信息物理系统模型;其次,提出了一种从平行Petri网到赋时Petri网的抽象简化方法,大规模压缩优化调度所需搜索的状态空间;再次,定义了策略Petri网以描述最优调度策略.最后,给出了平行Petri网与策略Petri网同步执行算法,使得平行Petri网与物理系统同步执行.     

Timed Petri Nets in Hybrid Systems: Stability and Supervisory Control

In this paper, timed Petri nets are used to model and control hybrid systems. Petri nets are used instead of finite automata primarily because of the advantages they offer in dealing with concurrency and complexity issues. A brief overview of existing results on hybrid systems that are based on Petri nets is first presented. A class of timed Petri nets named programmable timed Petri nets (PTPN) is then used to model hybrid systems. Using the PTPN, the stability and supervisory control of hybrid systems are addressed and efficient algorithms are introduced. In particular, we present sufficient conditions for the uniform ultimate boundness of hybrid systems composed of multiple linear time invariant plants which are switched between using a logical rule described by a Petri net. This paper also examines the supervisory control of a hybrid system in which the continuous state is transfered to a region of the state space in a way that respects safety specifications on the plant's discrete and continuous dynamics.     

Modeling, controlling, and simulation of local area networks for flexible manufacturing systems using Petri nets

Flexible manufacturing systems (FMSs) are advanced manufacturing systems comprising machining cells, robots, a automated guided vehicles (AGVs) that function under the hierarchical control of computers. The flexibility of an FMS primarily imparted by integrating the functions of different system elements such as machining cells, robots, AGVs usig computers. In this paper, the role of local area networks (LANs) in FMSs is explicated. The contribution of this paper is twofold. First, the differences between the LANs in conventional systems (CLANs) to that of LANs in FMSs (FLANs) are explicated. Secondly, a new class of Petri nets (PNs) called augmented timed PNs are introduced as a modeling tool designing, simulating, analyzing, and finally implementing the LAN in FMS. A software package has been developed to simulate the Petri net models (PNMs). The software package developed aids system designers by simulating the PNM and generating certain performance criteria.     

A heterogeneous modular robotic design for fast response to a diversity of tasks

This paper describes a heterogeneous modular robot system design which attempts to give a quick solution to a diversity of tasks. The approach is based on the use of an inventory of three types of modules i.e., power and control module, joint module and specialized module. Each module type aims to balance versatility and functionality. Their design permits rapid and cost effective design and fabrication. They are interchangeable in different ways to form different robot or system configurations. Depending on the task, the operator decides what type of robot can provide the best performance within the mission. A spherical joint module is described and used to build different robots, hence, forward and inverse kinematics models are obtained. Finally, from the modules described in this work, several robot configurations such as robotic arms, leg-based robots and wheel-based robots are assembled to demonstrate the execution of manipulation and locomotion tasks.     

Coloured Petri Nets and CPN Tools for modelling and validation of concurrent systems

Coloured Petri Nets (CPNs) is a language for the modelling and validation of systems in which concurrency, communication, and synchronisation play a major role. Coloured Petri Nets is a discrete-event modelling language combining Petri nets with the functional programming language Standard ML. Petri nets provide the foundation of the graphical notation and the basic primitives for modelling concurrency, communication, and synchronisation. Standard ML provides the primitives for the definition of data types, describing data manipulation, and for creating compact and parameterisable models. A CPN model of a system is an executable model representing the states of the system and the events (transitions) that can cause the system to change state. The CPN language makes it possible to organise a model as a set of modules, and it includes a time concept for representing the time taken to execute events in the modelled system. CPN Tools is an industrial-strength computer tool for constructing and analysing CPN models. Using CPN Tools, it is possible to investigate the behaviour of the modelled system using simulation, to verify properties by means of state space methods and model checking, and to conduct simulation-based performance analysis. User interaction with CPN Tools is based on direct manipulation of the graphical representation of the CPN model using interaction techniques, such as tool palettes and marking menus. A license for CPN Tools can be obtained free of charge, also for commercial use.     

Semantics for consistent activation in context-oriented systems

ContextContext-oriented programming languages provide dedicated programming abstractions to define behavioral adaptations and means to combine those adaptations dynamically according to sensed context changes. Some of these languages feature programming abstractions to explicitly define interaction dependencies among contexts. However, the semantics of context activation and the meaning of dependency relations have been described only informally, which in some cases has led to incorrect specifications, faulty implementations and inconsistent system behavior.ObjectiveWith the aim of avoiding faulty implementations and inconsistencies during system execution, this paper proposes both a formal and run-time model of contexts, context activation and context interaction.MethodAs a formal and computational basis, we introduce context Petri nets, a model based on Petri nets, which we found to match closely the structure of contexts in context-oriented systems. The operational semantics of Petri nets permits the modeling of run-time context activations. Existing Petri net analyses allow us to reason about system properties. As validation, we carried out small and medium-sized case studies.ResultsIn the cases explored, context Petri nets served effectively as underlying run-time model to ensure that declared context interaction constraints remain consistent during context manipulation. Moreover, context Petri nets enabled us to analyze certain properties regarding the activation state of particular contexts.ConclusionContext Petri nets thus proved to be appropriate to encode and manage the semantics of context activation, both formally and computationally, so as to preserve the consistency of context-oriented systems.     

Operations management issues in design and control of hybrid human-robot collaborative manufacturing systems: a survey

A manufacturing system able to perform a high variety of tasks requires different types of resources. Fully automated systems using robots possess high speed, accuracy, tirelessness, and force, but they are expensive. On the other hand, human workers are intelligent, creative, flexible, and able to work with different tools in different situations. A combination of these resources forms a human-machine/robot (hybrid) system, where humans and robots perform a variety of tasks (manual, automated, and hybrid tasks) in a shared workspace. Contrarily to the existing surveys, this study is dedicated to operations management problems (focusing on the applications and features) for human and machine/robot collaborative systems in manufacturing. This research is divided into two types of interactions between human and automated components in manufacturing and assembly systems: dual resource constrained (DRC) and human-robot collaboration (HRC) optimization problems. Moreover, different characteristics of the workforce and machines/robots such as heterogeneity, homogeneity, ergonomics, and flexibility are introduced. Finally, this paper identifies the optimization challenges and problems for hybrid systems. The existing literature on HRC focuses mainly on the robotic point of view and not on the operations management and optimization aspects. Therefore, the future research directions include the design of models and methods to optimize HRC systems in terms of ergonomics, safety, and throughput. In addition, studying flexibility and reconfigurability in hybrid systems is one of the main research avenues for future research.