Petri Net Plans

Programming the behavior of multi-robot systems is a challenging task which has a key role in developing effective systems in many application domains. In this paper, we present Petri Net Plans (PNPs), a language based on Petri Nets (PNs), which allows for intuitive and effective robot and multi-robot behavior design. PNPs are very expressive and support a rich set of features that are critical to develop robotic applications, including sensing, interrupts and concurrency. As a central feature, PNPs allow for a formal analysis of plans based on standard PN tools. Moreover, PNPs are suitable for modeling multi-robot systems and the developed behaviors can be executed in a distributed setting, while preserving the properties of the modeled system. PNPs have been deployed in several robotic platforms in different application domains. In this paper, we report three case studies, which address complex single robot plans, coordination and collaboration.     

Efficient behavior learning in human–robot collaboration

We present a novel method for a robot to interactively learn, while executing, a joint human–robot task. We consider collaborative tasks realized by a team of a human operator and a robot helper that adapts to the human’s task execution preferences. Different human operators can have different abilities, experiences, and personal preferences so that a particular allocation of activities in the team is preferred over another. Our main goal is to have the robot learn the task and the preferences of the user to provide a more efficient and acceptable joint task execution. We cast concurrent multi-agent collaboration as a semi-Markov decision process and show how to model the team behavior and learn the expected robot behavior. We further propose an interactive learning framework and we evaluate it both in simulation and on a real robotic setup to show the system can effectively learn and adapt to human expectations.     

Integrated Premission Planning and Execution for Unmanned Ground Vehicles

Fielding robots in complex applications can stress the human operators responsible for supervising them, particularly because the operators might understand the applications but not the details of the robots. Our answer to this problem has been to insert agent technology between the operator and the robotic platforms. In this paper, we motivate the challenges in defining, developing, and deploying the agent technology that provides the glue in the application of tasking unmanned ground vehicles in a military setting. We describe how a particular suite of architectural components serves equally well to support the interactions between the operator, planning agents, and robotic agents, and how our approach allows autonomy during planning and execution of a mission to be allocated among the human and artificial agents. Our implementation and demonstrations (in real robots and simulations) of our multi-agent system shows significant promise for the integration of unmanned vehicles in military applications.     

Decentralized planning and control for UAV–UGV cooperative teams

In this paper we study a symbiotic aerial vehicle-ground vehicle robotic team where unmanned aerial vehicles (UAVs) are used for aerial manipulation tasks, while unmanned ground vehicles (UGVs) aid and assist them. UGV can provide a UAV with a safe landing area and transport it across large distances, while UAV can provide an additional degree of freedom for the UGV, enabling it to negotiate obstacles. We propose an overall system control framework that includes high-accuracy motion planning for each individual robot and ad-hoc decentralized mission planning for complex missions. Experimental results obtained in a mockup arena for parcel transportation scenario show that the system is able to plan and execute missions in various environments and that the obtained plans result in lower energy consumption.     

Model-driven design space exploration for multi-robot systems in simulation

Multi-robot systems are increasingly deployed to provide services and accomplish missions whose complexity or cost is too high for a single robot to achieve on its own. Although multi-robot systems offer increased reliability via redundancy and enable the execution of more challenging missions, engineering these systems is very complex. This complexity affects not only the architecture modelling of the robotic team but also the modelling and analysis of the collaborative intelligence enabling the team to complete its mission. Existing approaches for the development of multi-robot applications do not provide a systematic mechanism for capturing these aspects and assessing the robustness of multi-robot systems. We address this gap by introducing ATLAS, a novel model-driven approach supporting the systematic design space exploration and robustness analysis of multi-robot systems in simulation. The ATLAS domain-specific language enables modelling the architecture of the robotic team and its mission and facilitates the specification of the team’s intelligence. We evaluate ATLAS and demonstrate its effectiveness in three simulated case studies: a healthcare Turtlebot-based mission and two unmanned underwater vehicle missions developed using the Gazebo/ROS and MOOS-IvP robotic platforms, respectively.

     

Operator Performance in Exploration Robotics

Mobile robots can accomplish high-risk tasks without exposing humans to danger: robots go where humans fear to tread. Until the time in which completely autonomous robots are fully deployed, remote operators will be required in order to fulfill desired missions. Remotely controlling a robot requires that the operator receives the information about the robot??s surroundings, as well as its location in the scenario. Based on a set of experiments conducted with users, we evaluate the performance of operators when they are provided with a hand-held-based interface or a desktop-based interface. Results show how performance depends on the task asked of the operator and the scenario in which the robot is moving. The conclusions prove that the operator??s intra-scenario mobility when carrying a hand-held device can counterbalance the limitations of the device. By contrast, the experiments show that if the operator cannot move inside of the scenario, his performance is significantly better when using a desktop-based interface. These results set the basis for a transfer of control policy in missions involving a team of operators, some equipped with hand-held devices and others working remotely with desktop-based computers.     

Interrupts

Some recent papers have included eulogies of the PDP-11 architecture, and of its interrupt mechanism in particular. This is dangerous if it is to set a standard for the coming decade. This note presents the view that all interrupts should be serviced by dedicated processes. However, the feasibility of this is determined by machine architectures, not just by the hardware interrupt mechanism itself. Considering first the well-known PDP-11, we then summarize the origin and function of interrupts before stating a philosophy for interrupt-handling. It is then shown, with reference to the PDP-11, how not only the interrupt mechanism but also features such as stacks and procedure mechanisms may affect this philosophy. We conclude by suggesting alternative architectural features to unify interrupts, procedure calls, process invocations, and interprocess communication, as will be required for the support of recent and future languages.     

An integrated system for planning,navigation and robotic assistance for mandible reconstruction surgery

Mandibular reconstruction is an extremely complex and high-risk surgery. The aim of the study was to integrate robotics technology into mandibular reconstruction surgery as it can reduce surgeons’ workload and improve the accuracy and quality of the surgery. In this study, we first introduced a mandibular reconstruction surgery robotic system, which includes integrated surgery planning, an optical navigation system, and a robot-assisted operation. Second, we addressed novel mandibular reconstruction surgery that is aided by a multi-arm robot with an optical navigation system. Finally, we conducted an accuracy test, a skull model experiment, and an animal experiment to evaluate the robotic system. Experiments showed that the robot’s mean placement error was 1.0205 mm, which was acceptable for clinical application. We will continue to study this point, however, and plan to reduce the error eventually to 1.00 mm. The robot ran smoothly and accurately. It was also able to perform fibular segment implantation, positioning, and auxiliary fixation during mandibular reconstruction surgery.     

Planning as satisfiability: parallel plans and algorithms for plan search

We address two aspects of constructing plans efficiently by means of satisfiability testing: efficient encoding of the problem of existence of plans of a given number t of time points in the propositional logic and strategies for finding plans, given these formulae for different values of t.For the first problem we consider three semantics for plans with parallel operator application in order to make the search for plans more efficient. The standard semantics requires that parallel operators are independent and can therefore be executed in any order. We consider a more relaxed definition of parallel plans which was first proposed by Dimopoulos et al., as well as a normal form for parallel plans that requires every operator to be executed as early as possible. We formalize the semantics of parallel plans emerging in this setting and present translations of these semantics into the propositional logic. The sizes of the translations are asymptotically optimal. Each of the semantics is constructed in such a way that there is a plan following the semantics exactly when there is a sequential plan, and moreover, the existence of a parallel plan implies the existence of a sequential plan with as many operators as in the parallel one.For the second problem we consider strategies based on testing the satisfiability of several formulae representing plans of n time steps for several values of n concurrently by several processes. We show that big efficiency gains can be obtained in comparison to the standard strategy of sequentially testing the satisfiability of formulae for an increasing number of time steps.     

Decentralized Coordinated Tracking with Mixed Discrete–Continuous Decisions

In this paper, we study the problem of dynamically positioning a team of mobile robots for target tracking. We treat the coordination of mobile robots for target tracking as a joint team optimization to minimize uncertainty in target state estimates over a fixed horizon. The optimization is inherently a function of both the positioning of robots in continuous space and the assignment of robots to targets in discrete space. Thus, the robot team must make decisions over discrete and continuous variables. In contrast to methods that decouple target assignments and robot positioning, our approach avoids the strong assumption that a robot's utility for observing a target is independent of other robots’ observations. We formulate the optimization as a mixed integer nonlinear program and apply integer relaxation to develop an approximate solution in decentralized form. We demonstrate our coordinated multirobot tracking algorithm both in simulation and using a pair of mobile robotic sensor platforms to track moving pedestrians. Our results show that coupling target assignment and robot positioning realizes coordinated behaviors that are not possible with decoupled methods.     

Structural plan similarity based on refinements in the space of partial plans

Plan similarity measures play a key role in many areas of artificial intelligence, such as case‐based planning, plan recognition, ambient intelligence, or digital storytelling. In this paper, we present 2 novel structural similarity measures to compare plans based on a search process in the space of partial plans. Partial plans are compact representations of sets of plans with some common structure and can be organized in a lattice so that the most general partial plans are above the most specific ones. To compute our similarity measures, we traverse this space of partial plans from the most general to the most specific using successive refinements. Our first similarity measure is designed for propositional plan formalisms, and the second is designed for classical planning formalisms (including variables and types). We also introduce 2 novel refinement operators used to traverse the space of plans: an ideal downward refinement operator for propositional partial plans and a finite and complete downward refinement operator for classical partial plans. Finally, we evaluate our similarity measures in the context of a nearest neighbor classifier using 2 datasets commonly used in the plan recognition literature (Linux and Monroe), showing good results in both synthetic and real data.     

中断驱动系统模型检验

针对一类中断驱动系统提出了一种建模和模型检验的方法.该系统通常由中断处理程序和操作系统调度的任务组成,前者由中断源触发后处理中断事件,后者则负责处理系统的日常任务以及某些中断处理事件的后续处理.因为这类系统是实时控制系统,对中断事件的处理需要在规定时间内响应并完成,否则可能造成严重的系统失效.为了帮助系统设计人员在系统设计过程中应用模型检验技术来提高系统的正确性,首先确定了此类系统中与时序性质相关的系统要素(包括系统调度任务、中断源、中断处理程序)和相关参数,并要求设计人员在设计阶段明确指出这些要素的参数.然后,提出了将这些要素和参数自动转化为形式化模型的方法:使用时间自动机对中断事件进行建模,使用中断向量表和CPU处理栈对中断处理过程进行建模.对于得到的形式化模型,给出了针对中断处理超时错误的检测方法,并在此基础上给出了针对共享资源的完整性、子程序原子性的检验方法.     

Autonomous grasp and manipulation planning using a ToF camera

A time-of-flight camera can help a service robot to sense its 3D environment. In this paper, we introduce our methods for sensor calibration and 3D data segmentation to use it to automatically plan grasps and manipulation actions for a service robot. Impedance control is intensively used to further compensate the modeling error and to apply the computed forces. The methods are further demonstrated in three service robotic applications. Sensor-based motion planning allows the robot to move within dynamic and cluttered environment without collision. Unknown objects can be detected and grasped. In the autonomous ice cream serving scenario, the robot captures the surface of ice cream and plans a manipulation trajectory to scoop a portion of ice cream.     

Assisted teleoperation in changing environments with a mixture of virtual guides

Haptic guidance is a powerful technique to combine the strengths of humans and autonomous systems for teleoperation. The autonomous system can provide haptic cues to enable the operator to perform precise movements; the operator can interfere with the plan of the autonomous system leveraging his/her superior cognitive capabilities. However, providing haptic cues such that the individual strengths are not impaired is challenging because low forces provide little guidance, whereas strong forces can hinder the operator in realizing his/her plan. Based on variational inference, we learn a Gaussian mixture model (GMM) over trajectories to accomplish a given task. The learned GMM is used to construct a potential field which determines the haptic cues. The potential field smoothly changes during teleoperation based on our updated belief over the plans and their respective phases. Furthermore, new plans are learned online when the operator does not follow any of the proposed plans or after changes in the environment. User studies confirm that our framework helps users perform teleoperation tasks more accurately than without haptic cues and, in some cases, faster. Moreover, we demonstrate the use of our framework to help a subject teleoperate a 7 DoF manipulator in a pick-and-place task.     

Persistent surveillance for unmanned aerial vehicles subject to charging and temporal logic constraints

In this work, we present a novel method for automating persistent surveillance missions involving multiple vehicles. Automata-based techniques are used to generate collision-free motion plans for a team of vehicles to satisfy a temporal logic specification. Vector fields are created for use with a differential flatness-based controller, allowing vehicle flight and deployment to be fully automated according to the motion plans. The use of charging platforms with the vehicles allows for truly persistent missions. Experiments were performed with two quadrotors for two different missions over 50 runs each to validate the theoretical results.     

Adapting plans in progress in distributed supervisory work: aspects of complexity, coupling, and control

Distributed supervisory control systems often rely on complex and centralized plans to cope with a variety of unanticipated situations. Replanning requires practitioners to forgo standard procedures in favor of making simple plans without simplifying, managing task coupling, and anticipating team needs to provide decentralized and elaborate plans. This article proposes a plan classification scheme to study what features of plans facilitate or hinder adaptation and a framework to examine how features of plans influence the cognitive processes of replanning. The plan features have been assigned to the categories of plan complexity, coupling, and control. Plans are task networks sharing similar features of complexity and coupling to technical systems. The proposed framework sets out to explore how plan features influence the processes of recognizing plan disruptions, reviewing challenges and different team stances, repairing plans to resolve new risks, and reacting by coordinating team efforts to execute plans. The framework draws on the Extended Control Model (ECOM) to integrate the four processes of replanning into a set of control loops. The benefits of this framework are illustrated in the context of operator training and decision support.     

Robust variable admittance control for human–robot co-manipulation of objects with unknown load

Over the last years, physical Human–Robot Interaction (pHRI) has become a particularly interesting topic for industrial tasks. An important issue is allowing people and robots to collaborate in an useful, simple and safe manner. In this work, we propose a new framework that allows the person to collaborate with a robot manipulator, while the robot has its own predefined task. To allow the robot to smoothly switch from its own task to be a compliant collaborator for the person, a variable admittance control is developed. Furthermore, in general the task to accomplish requires the robot to carry variable, unknown, loads at the end-effector. To include this feature in our framework, a robust control is also included to preserve the performance of the robot despite uncertainties coming from the unknown load. To validate our approach, experiments were carried out with a Kuka LBR iiwa 14 R820, first to validate both parts of the controller, and finally, to study a use-case scenario similar to an industrial production line. Results show the efficiency of this approach to allow the person to collaborate at any moment while the robot is capable of performing another task. This flexible framework for object co-manipulation also allows unknown loads up to 2 kg to be handled without making the task more difficult for the person.     

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.     

基于AADL的中断控制设计方法

介绍了结构分析与设计语言(AADL)的应用研究和优点,以某航天器控制分系统部分中断类型为例,针对AADL语言自身直接对中断问题描述能力上的不足,提出了两种基于AADL的中断控制设计方法,并进行了详细阐述和比较,为系统设计阶段对中断控制的抽象描述提供了一种思路和方法,从而有利于实现早期阶段对模型可调度性等方面验证的途径.     

An instrumented glove for grasp specification invirtual-reality-based point-and-direct telerobotics

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping “flavors” has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as “put that there” cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest