空间桁架的人-机协作装配规划与试验验证

该文以空间直立桁架结构的人-机协作装配流程为研究对象,提出一种适用于径向快装桁架模块单元的创新构型,建立了基于状态矩阵和邻接矩阵的桁架结构装配序列、装配模式和装配过程的数学模型。该文还对空间环境下桁架装配的人-机能力约束进行分析,提出基于动素分析的装配任务层级化分解方法,利用比较分配原则制定了适用编程的人-机协作装配任务流程和分配方案。基于所提出方案,通过开展人穿戴模拟宇航服与机械臂协作装配 5 m 长直立桁架结构的地面演示试验,验证了该方案的合理性和组装流程的可行性,为空间大型设施在轨构建提供了技术原理支撑。     

Human-robot contactless collaboration with mixed reality interface

A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.     

满足不同交互任务的人机共融系统设计

人与机器人共同协作的灵活生产模式已经成为工业成产的迫切需求,因此,近年来人机共融系统方面的研究受到了越来越多关注.设计并实现了一种满足不同交互任务的人机共融系统,人体动作的估计和机器人的交互控制是其中的关键技术.首先,提出了一种基于多相机和惯性测量单元信息融合的人体姿态解算方法,通过构造优化问题,融合多相机下的2D关节检测信息和所佩戴的惯性测量单元测量信息,对人体运动学姿态进行优化估计,改善了单一传感器下,姿态信息不全面以及对噪声敏感的问题,提升了姿态估计的准确度.其次,结合机器人的运动学特性和人机交互的特点,设计了基于目标点跟踪和模型预测控制的机器人控制策略,使得机器人能够通过调整控制参数,适应动态的环境和不同的交互需求,同时保证机器人和操作人员的安全.最后,进行了动作跟随、物品传递、主动避障等人机交互实验,实验结果表明了所设计的机器人交互系统在人机共融环境下的有效性和可靠性.     

How to include User eXperience in the design of Human-Robot Interaction

In recent years Human-Robot Collaboration (HRC) has become a strategic research field, considering the emergent need for common collaborative execution of manufacturing tasks, shared between humans and robots within the modern factories. However, the majority of the research focuses on the technological aspects and enabling technologies, mainly directing to the robotic side, and usually neglecting the human factors. This work deals with including the needs of the humans interacting with robots in the design in human-robot interaction (HRI). In particular, the paper proposes a user experience (UX)-oriented structured method to investigate the human-robot dialogue to map the interaction with robots during the execution of shared tasks, and to finally elicit the requirements for the design of valuable HRI. The research adopted the proposed method to an industrial case focused on assembly operations supported by collaborative robots and AGVs (Automated Guided Vehicles). A multidisciplinary team was created to map the HRI for the specific case with the final aim to define the requirements for the design of the system interfaces. The novelty of the proposed approach is the inclusion of typically interaction design tools focusing in the analysis of the UX into the design of the system components, without merely focusing on the technological issues. Experimental results highlighted the validity of the proposed method to identify the interaction needs and to drive the interface design.     

Fast scheduling of human-robot teams collaboration on synchronised production-logistics tasks in aircraft assembly

The deployment of human-robot teams (HRTs) promises to realise the potential of each team member regarding their distinct abilities and combines efficiency and flexibility in manufacturing operations. However, enabling effective coordination amongst collaborative tasks performed by humans and robots while ensuring safety and satisfying specific constraints is challenging. Motivated by real-world applications that Boeing and Airbus adopt HRTs in manufacturing operations, this paper investigates the allocating and coordinating of HRTs to support safe and efficient human-robot collaboration on synchronised production-logistics tasks in aircraft assembly. We connect the operations research and robotics communities by formulating the problem with precedence constraints, spatial constraints, temporal constraints, and synchronisation constraints that fits within the classic multi-robot task allocation (MRTA) category into a flexible job shop scheduling problem. Two exact approaches, including mixed-integer linear programming (MILP) and constraint programming (CP), are proposed to formulate and solve this problem. A benchmark set with 80 instances (e.g., small/medium-scale and large-scale instances) that corresponds to real dimensions of industrial problems with production tasks, subtasks, locations, deadlines, human worker eligibility and capacity, robot eligibility and capacity, material handling system capacity, and travel times is developed. Experimental evaluation with a total of 1200 independent tests on the benchmark set shows the superiority of the CP approach comparing the MILP approach for efficiently solving real-life scheduling problems of HRTs collaboration on synchronised production-logistics tasks in aircraft assembly.     

A digital twin-driven human-robot collaborative assembly approach in the wake of COVID-19

In the wake of COVID-19, the production demand of medical equipment is increasing rapidly. This type of products is mainly assembled by hand or fixed program with complex and flexible structure. However, the low efficiency and adaptability in current assembly mode are unable to meet the assembly requirements. So in this paper, a new framework of human-robot collaborative (HRC) assembly based on digital twin (DT) is proposed. The data management system of proposed framework integrates all kinds of data from digital twin spaces. In order to obtain the HRC strategy and action sequence in dynamic environment, the double deep deterministic policy gradient (D-DDPG) is applied as optimization model in DT. During assembly, the performance model is adopted to evaluate the quality of resilience assembly. The proposed framework is finally validated by an alternator assembly case, which proves that DT-based HRC assembly has a significant effect on improving assembly efficiency and safety.     

Collision-free human-robot collaboration based on context awareness

Recent advancements in human-robot collaboration have enabled human operators and robots to work together in a shared manufacturing environment. However, current distance-based collision-free human-robot collaboration system can only ensure human safety but not assembly efficiency. In this paper, the authors present a context awareness-based collision-free human-robot collaboration system that can provide human safety and assembly efficiency at the same time. The system can plan robotic paths that avoid colliding with human operators while still reach target positions in time. Human operators’ poses can also be recognised with low computational expenses to further improve assembly efficiency. To support the context-aware collision-free system, a complete collision sensing module with sensor calibration algorithms is proposed and implemented. An efficient transfer learning-based human pose recognition algorithm is also adapted and tested. Two experiments are designed to test the performance of the proposed human pose recognition algorithm and the overall system. The results indicate an efficiency improvement of the overall system.     

A Minimum-Jerk Speed-Planning Algorithm for Coordinated Planning and Control of Automated Assembly Manufacturing

We previously proposed a general algorithm for coordinating the motions among multiple machines in a shared assembly environment based on a constant-speed motion model. In this paper, we extend this work to a minimum-jerk polynomial motion model and describe a new speed-planning algorithm to plan automated assembly machines' motions. Machines are planned sequentially, based on their priorities, by mapping the motions of higher-priority machines into forbidden regions in two-dimensional space-time graphs. Collision-free minimum-jerk motions are then planned between the forbidden regions in the graphs. The new speed-planning algorithm is evaluated on a dual-robot surface-mount technology assembly machine in which both robots share a common workspace. Note to Practitioners—Automated assembly processes, especially surface-mount technology manufacturing, require a high degree of precision when placing certain components. This motivated us to find a way of maintaining good positional accuracy by planning smooth motions for the machines that perform these tasks. Since many of these machines have two or more robots, their motions must also be coordinated. We developed an algorithm that combines coordinated motion concepts with a minimum-jerk motion model that can solve these problems. The algorithm plans segmented paths for the robots and then sequentially plans their speeds to prevent collisions between them. The planned speeds ensure position, velocity, and acceleration continuity between path segments. The smooth motions resulting from this method enable high-accuracy component placement. The tradeoff for this improvement is increased cycle time compared to other speed-planning methods.     

Operations management issues in design and control of hybrid human-robot collaborative manufacturing systems: a survey

A manufacturing system able to perform a high variety of tasks requires different types of resources. Fully automated systems using robots possess high speed, accuracy, tirelessness, and force, but they are expensive. On the other hand, human workers are intelligent, creative, flexible, and able to work with different tools in different situations. A combination of these resources forms a human-machine/robot (hybrid) system, where humans and robots perform a variety of tasks (manual, automated, and hybrid tasks) in a shared workspace. Contrarily to the existing surveys, this study is dedicated to operations management problems (focusing on the applications and features) for human and machine/robot collaborative systems in manufacturing. This research is divided into two types of interactions between human and automated components in manufacturing and assembly systems: dual resource constrained (DRC) and human-robot collaboration (HRC) optimization problems. Moreover, different characteristics of the workforce and machines/robots such as heterogeneity, homogeneity, ergonomics, and flexibility are introduced. Finally, this paper identifies the optimization challenges and problems for hybrid systems. The existing literature on HRC focuses mainly on the robotic point of view and not on the operations management and optimization aspects. Therefore, the future research directions include the design of models and methods to optimize HRC systems in terms of ergonomics, safety, and throughput. In addition, studying flexibility and reconfigurability in hybrid systems is one of the main research avenues for future research.     

A Human Aware Mobile Robot Motion Planner

Robot navigation in the presence of humans raises new issues for motion planning and control when the humans must be taken explicitly into account. We claim that a human aware motion planner (HAMP) must not only provide safe robot paths, but also synthesize good, socially acceptable and legible paths. This paper focuses on a motion planner that takes explicitly into account its human partners by reasoning about their accessibility, their vision field and their preferences in terms of relative human-robot placement and motions in realistic environments. This planner is part of a human-aware motion and manipulation planning and control system that we aim to develop in order to achieve motion and manipulation tasks in the presence or in synergy with humans.     

A Human-Robot Collaboration Framework for Improving Ergonomics During Dexterous Operation of Power Tools

In this work, we present a novel control approach to human-robot collaboration that takes into account ergonomic aspects of the human co-worker during power tool operations. The method is primarily based on estimating and reducing the overloading torques in the human joints that are induced by the manipulated external load. The human overloading joint torques are estimated and monitored using a whole-body dynamic state model. The appropriate robot motion that brings the human into the suitable ergonomic working configuration is obtained by an optimisation method that minimises the overloading joint torques. The proposed optimisation process includes several constraints, such as the human arm muscular manipulability and safety of the collaborative task, to achieve a task-relevant optimised configuration. We validated the proposed method by a user study that involved a human-robot collaboration task, where the subjects operated a polishing machine on a part that was brought to them by the collaborative robot. A statistical analysis of ten subjects as an experimental evaluation of the proposed control framework is provided to demonstrate the potential of the proposed control framework in enabling ergonomic and task-optimised human-robot collaboration.     

A collaborative approach to assembly sequence planning

Distributed product development requires collaborative work among team members. For the sake of supporting assembly planning activities involving geographically dispersed designers, this paper presents an approach of collaborative assembly sequence planning to validate the assemblability of parts and subassemblies rapidly. In order to increase the planning efficiency and support the collaborative planning, role-based model is exploited to compress or simplify the product. In role-based model, the B-rep models are simplified according to the permissions associated with the role, so the surfaces invisible from outside of the model are removed. In collaborative planning, the planning tasks are assigned to different designers that carry out the collaborative planning, respectively. In this paper, a knowledge-based approach is proposed to the assembly sequence planning problem. This research shows that the typical or standard CSBAT (Connection Semantics Based Assembly Tree) can be applied to a given assembly problem. This paper presents the structure of the Co-ASP (Collaborative Assembly Sequence Planning System) and provides an example to illustrate the collaborative planning approach.     

Control sharing in human-robot team interaction

The interaction between humans and robot teams is highly relevant in many application domains, for example in collaborative manufacturing, search and rescue, and logistics. It is well-known that humans and robots have complementary capabilities: Humans are excellent in reasoning and planning in unstructured environments, while robots are very good in performing tasks repetitively and precisely. In consequence, one of the key research questions is how to combine human and robot team decision making and task execution capabilities in order to exploit their complementary skills. From a controls perspective this question boils down to how control should be shared among them. This article surveys advances in human-robot team interaction with special attention devoted to control sharing methodologies. Additionally, aspects affecting the control sharing design, such as human behavior modeling, level of autonomy and human-machine interfaces are identified. Open problems and future research directions towards joint decision making and task execution in human-robot teams are discussed.     

Digital twins for collaborative robots: A case study in human-robot interaction

Human-robot collaboration (HRC) can expand the level of automation in areas that have conventionally been difficult to automate such as assembly. However, the need of adaptability and the dynamics of human presence are keeping the full potential of human-robot collaborative systems difficult to achieve. This paper explores the opportunities of using a digital twin to address the complexity of collaborative production systems through an industrial case and a demonstrator. A digital twin, as a virtual counterpart of a physical human-robot assembly system, is built as a ‘front-runner’ for validation and control throughout its design, build and operation. The forms of digital twins along system's life cycle, its building blocks and the potential advantages are presented and discussed. Recommendations for future research and practice in the use of digital twins in the field of cobotics are given.     

Robust human position estimation in cooperative robotic cells

Robots are increasingly present in our lives, sharing the workspace and tasks with human co-workers. However, existing interfaces for human-robot interaction / cooperation (HRI/C) have limited levels of intuitiveness to use and safety is a major concern when humans and robots share the same workspace. Many times, this is due to the lack of a reliable estimation of the human pose in space which is the primary input to calculate the human-robot minimum distance (required for safety and collision avoidance) and HRI/C featuring machine learning algorithms classifying human behaviours / gestures. Each sensor type has its own characteristics resulting in problems such as occlusions (vision) and drift (inertial) when used in an isolated fashion. In this paper, it is proposed a combined system that merges the human tracking provided by a 3D vision sensor with the pose estimation provided by a set of inertial measurement units (IMUs) placed in human body limbs. The IMUs compensate the gaps in occluded areas to have tracking continuity. To mitigate the lingering effects of the IMU offset we propose a continuous online calculation of the offset value. Experimental tests were designed to simulate human motion in a human-robot collaborative environment where the robot moves away to avoid unexpected collisions with de human. Results indicate that our approach is able to capture the human’s position, for example the forearm, with a precision in the millimetre range and robustness to occlusions.     

Human-robot collaboration while sharing production activities in dynamic environment: SPADER system

Interactive robot doing collaborative work in hybrid work cell need adaptive trajectory planning strategy. Indeed, systems must be able to generate their own trajectories without colliding with dynamic obstacles like humans and assembly components moving inside the robot workspace. The aim of this paper is to improve collision-free motion planning in dynamic environment in order to insure human safety during collaborative tasks such as sharing production activities between human and robot. Our system proposes a trajectory generating method for an industrial manipulator in a shared workspace. A neural network using a supervised learning is applied to create the waypoints required for dynamic obstacles avoidance. These points are linked with a quintic polynomial function for smooth motion which is optimized using least-square to compute an optimal trajectory. Moreover, the evaluation of human motion forms has been taken into consideration in the proposed strategy. According to the results, the proposed approach is an effective solution for trajectories generation in a dynamic environment like a hybrid workspace.     

Inverse Kinematics Based Human Mimicking System using Skeletal Tracking Technology

Humanoid robots needs to have human-like motions and appearance in order to be well-accepted by humans. Mimicking is a fast and user-friendly way to teach them human-like motions. However, direct assignment of observed human motions to robot’s joints is not possible due to their physical differences. This paper presents a real-time inverse kinematics based human mimicking system to map human upper limbs motions to robot’s joints safely and smoothly. It considers both main definitions of motion similarity, between end-effector motions and between angular configurations. Microsoft Kinect sensor is used for natural perceiving of human motions. Additional constraints are proposed and solved in the projected null space of the Jacobian matrix. They consider not only the workspace and the valid motion ranges of the robot’s joints to avoid self-collisions, but also the similarity between the end-effector motions and the angular configurations to bring highly human-like motions to the robot. Performance of the proposed human mimicking system is quantitatively and qualitatively assessed and compared with the state-of-the-art methods in a human-robot interaction task using Nao humanoid robot. The results confirm applicability and ability of the proposed human mimicking system to properly mimic various human motions.     

Improving human robot collaboration through Force/Torque based learning for object manipulation

Human–Robot Collaboration (HRC) is a term used to describe tasks in which robots and humans work together to achieve a goal. Unlike traditional industrial robots, collaborative robots need to be adaptive; able to alter their approach to better suit the situation and the needs of the human partner. As traditional programming techniques can struggle with the complexity required, an emerging approach is to learn a skill by observing human demonstration and imitating the motions; commonly known as Learning from Demonstration (LfD). In this work, we present a LfD methodology that combines an ensemble machine learning algorithm (i.e. Random Forest (RF)) with stochastic regression, using haptic information captured from human demonstration. The capabilities of the proposed method are evaluated using two collaborative tasks; co-manipulation of an object (where the human provides the guidance but the robot handles the objects weight) and collaborative assembly of simple interlocking parts. The proposed method is shown to be capable of imitation learning; interpreting human actions and producing equivalent robot motion across a diverse range of initial and final conditions. After verifying that ensemble machine learning can be utilised for real robotics problems, we propose a further extension utilising Weighted Random Forest (WRF) that attaches weights to each tree based on its performance. It is then shown that the WRF approach outperforms RF in HRC tasks.     

Flexible telemanipulation based handy robot teaching on tape masking with complex geometry

Traditional robot teaching methods are cumbersome, tedious and difficult to scale for high-mix low-volume applications. The tape masking, a common process for surface protection before plasma spraying, spray painting and shot peening, is one of those domains where robotic automation lacks flexibility and reliability due to the complexity in task. Fortunately, it is still within the grasps of human-robot collaborative systems. This work presents a telemanipulation-based robot teaching framework that is able to let the robot manipulator cope with the taping tasks with complex workpiece geometries. The proposed framework allows quick calibration, variable motion mapping, and indexing so that the operators can easily set up and guide the robotic taping system to cover the tapes onto the layers and grooves of different workpieces. This framework enables the operators to change the motion mapping scale for both large-scale guidance and fine motion dexterous manipulation. Meanwhile, an indexing function makes it possible for the operators to re-map their poses from the edges of their comfortable regions. A portable VR system is applied in the telemanipulation system. With its six DoF motion precisely measured in real-time, the proposed motion remapping algorithms enable the operators to directly guide the robot in their selected scales. Experimental results show that the proposed framework facilitates robot programming on the manipulation of the complex workpieces that have multi-layer surfaces and grooves in between. It also reduces the teaching time comparing to other methods. This system and method improve teaching efficiency and convenience, which has potential value to be deployed in manufacturing.