Optimizing task scheduling in human-robot collaboration with deep multi-agent reinforcement learning

Human-Robot Collaboration (HRC) presents an opportunity to improve the efficiency of manufacturing processes. However, the existing task planning approaches for HRC are still limited in many ways, e.g., co-robot encoding must rely on experts’ knowledge and the real-time task scheduling is applicable within small state-action spaces or simplified problem settings. In this paper, the HRC assembly working process is formatted into a novel chessboard setting, in which the selection of chess piece move is used to analogize to the decision making by both humans and robots in the HRC assembly working process. To optimize the completion time, a Markov game model is considered, which takes the task structure and the agent status as the state input and the overall completion time as the reward. Without experts’ knowledge, this game model is capable of seeking for correlated equilibrium policy among agents with convergency in making real-time decisions facing a dynamic environment. To improve the efficiency in finding an optimal policy of the task scheduling, a deep-Q-network (DQN) based multi-agent reinforcement learning (MARL) method is applied and compared with the Nash-Q learning, dynamic programming and the DQN-based single-agent reinforcement learning method. A height-adjustable desk assembly is used as a case study to demonstrate the effectiveness of the proposed algorithm with different number of tasks and agents.     

Hybrid machine learning for human action recognition and prediction in assembly

As one of the critical elements for smart manufacturing, human-robot collaboration (HRC), which refers to goal-oriented joint activities of humans and collaborative robots in a shared workspace, has gained increasing attention in recent years. HRC is envisioned to break the traditional barrier that separates human workers from robots and greatly improve operational flexibility and productivity. To realize HRC, a robot needs to recognize and predict human actions in order to provide assistance in a safe and collaborative manner. This paper presents a hybrid approach to context-aware human action recognition and prediction, based on the integration of a convolutional neural network (CNN) and variable-length Markov modeling (VMM). Specifically, a bi-stream CNN structure parses human and object information embedded in video images as the spatial context for action recognition and collaboration context identification. The dependencies embedded in the action sequences are subsequently analyzed by a VMM, which adaptively determines the optimal number of current and past actions that need to be considered in order to maximize the probability of accurate future action prediction. The effectiveness of the developed method is evaluated experimentally on a testbed which simulates an assembly environment. High accuracy in both action recognition and prediction is demonstrated.     

A digital twin-driven human-robot collaborative assembly approach in the wake of COVID-19

In the wake of COVID-19, the production demand of medical equipment is increasing rapidly. This type of products is mainly assembled by hand or fixed program with complex and flexible structure. However, the low efficiency and adaptability in current assembly mode are unable to meet the assembly requirements. So in this paper, a new framework of human-robot collaborative (HRC) assembly based on digital twin (DT) is proposed. The data management system of proposed framework integrates all kinds of data from digital twin spaces. In order to obtain the HRC strategy and action sequence in dynamic environment, the double deep deterministic policy gradient (D-DDPG) is applied as optimization model in DT. During assembly, the performance model is adopted to evaluate the quality of resilience assembly. The proposed framework is finally validated by an alternator assembly case, which proves that DT-based HRC assembly has a significant effect on improving assembly efficiency and safety.     

A framework and method for Human-Robot cooperative safe control based on digital twin

Human-robot collaboration (HRC) combines the robot’s mechanical properties and predictability with human experience, logical thinking, and strain capabilities to alleviate production efficiency. However, ensuring the safety of the HRC process in-real time has become an urgent issue. Digital twin extends functions of virtual models in the design phase of the physical counterpart in the production phase through virtual-real interactive feedback, data fusion analysis, advanced computational features, etc. This paper proposes an HRC safety control framework and corresponding method based on the digital twin. In the design phase, virtual simulation and virtual reality technology are integrated to construct virtual twins of various HRC scenarios for testing and analyzing potential safety hazards. In the production phase, the safety distance between humans and robots of the HRC scene is monitored and calculated by an iterative algorithm according to machine vision and a convolutional neural network. Finally, the virtual twin is driven based on real-scene data, real-time online visual monitoring, and optimization of the HRC’s overall process. A case study using ABB-IRB1600 is presented to verify the feasibility of the proposed approach.     

光学头部姿态跟踪的多传感器数据融合研究

精确的头部姿态跟踪是室内增强现实系统实现高精度注册的关键技术之一. 本文介绍了使用传感器数据融合原理实现高精度的光学头部姿态跟踪的新方法. 该方法使用多传感器数据融合中的扩展卡尔曼滤波器和融合滤波器, 将两个互补的单摄像机Inside-out跟踪和双摄像机Outside-in跟踪的头部姿态进行数据融合, 以减小光学跟踪传感器的姿态误差. 设计了一个典型实验装置验证所提出的算法, 实验结果显示, 在静态测试下的姿态输出误差与使用误差协方差传播法则计算得到的结果是一致的; 在动态跟踪条件下, 与单个Inside-out或Outside-in跟踪相比, 所提出的光学头部姿态数据融合算法能够使跟踪器获得精度更高、更稳定的位置和方向信息.     

Difficulty and complexity definitions for assembly task allocation and assignment in human–robot collaborations: A review

This paper presents a literature review on the different aspects of task allocation and assignment problems in human–robot collaboration (HRC) tasks in industrial assembly environments. In future advanced industrial environments, robots and humans are expected to share the same workspace and collaborate to efficiently achieve shared goals. Difficulty- and complexity-aware HRC assembly is necessary for human-centric manufacturing, which is a goal of Industry 5.0. Therefore, the objective of this study is to clarify the definitions of difficulty and complexity used to encourage effective collaboration between humans and robots to leverage the adaptability of humans and the autonomy of robots. To achieve this goal, a systematic review of the following relevant databases for computer science was performed: IEEE Xplore, ScienceDirect, SpringerLink, ACM Digital Library, and ASME Digital Collection. The results extracted from 74 peer-reviewed research articles published until July 2022 were summarized and categorized into four taxonomies for 145 difficulty and complexity definitions from the perspectives of (1) definition-use objectives, (2) evaluation objectives, (3) evaluation factors, and (4) evaluation variables. Next, existing definitions were primarily classified according to the following two criteria to identify potential future studies on the formulation of new definitions for human-centric manufacturing: (1) agent specificity and (2) common aspects in manual and robotic assemblies.     

AR-based interaction for human-robot collaborative manufacturing

Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. This paper proposes a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup and a wearable AR gear (HoloLens). The workspace model and UIs are evaluated in a realistic diesel engine assembly task. The AR-based interactive UIs provide 21–24% and 57–64% reduction in the task completion and robot idle time, respectively, as compared to a baseline without interaction and workspace sharing. However, user experience assessment reveal that HoloLens based AR is not yet suitable for industrial manufacturing while the projector-mirror setup shows clear improvements in safety and work ergonomics.     

Three methodologies for the calibration of industrial manipulators: Experimental results on a SCARA robot

This article presents and compares three algorithms for the geometric parameter identification of industrial robots to increase its accuracy (static calibration). The estimation is based on the measure of the gripper pose errors when the robot follows suitable trajectories. The algorithms are general and can be applied to any robot providing that its kinematics is known. After a theoretical introduction to the general methodologies, these are applied to a selective compliance assembly robot arm (SCARA) robot analyzing its performance (precision, efficiency). Experimental results obtained with three methodologies are presented and discussed. The measure of the gripper pose error is based on a laser triangulation technique whose working principles are also recalled. © 2000 John Wiley & Sons, Inc.     

Collision-free human-robot collaboration based on context awareness

Recent advancements in human-robot collaboration have enabled human operators and robots to work together in a shared manufacturing environment. However, current distance-based collision-free human-robot collaboration system can only ensure human safety but not assembly efficiency. In this paper, the authors present a context awareness-based collision-free human-robot collaboration system that can provide human safety and assembly efficiency at the same time. The system can plan robotic paths that avoid colliding with human operators while still reach target positions in time. Human operators’ poses can also be recognised with low computational expenses to further improve assembly efficiency. To support the context-aware collision-free system, a complete collision sensing module with sensor calibration algorithms is proposed and implemented. An efficient transfer learning-based human pose recognition algorithm is also adapted and tested. Two experiments are designed to test the performance of the proposed human pose recognition algorithm and the overall system. The results indicate an efficiency improvement of the overall system.     

Human-object integrated assembly intention recognition for context-aware human-robot collaborative assembly

Human-robot collaborative (HRC) assembly combines the advantages of robot's operation consistency with human's cognitive ability and adaptivity, which provides an efficient and flexible way for complex assembly tasks. In the process of HRC assembly, the robot needs to understand the operator's intention accurately to assist the collaborative assembly tasks. At present, operator intention recognition considering context information such as assembly objects in a complex environment remains challenging. In this paper, we propose a human-object integrated approach for context-aware assembly intention recognition in the HRC, which integrates the recognition of assembly actions and assembly parts to improve the accuracy of the operator's intention recognition. Specifically, considering the real-time requirements of HRC assembly, spatial-temporal graph convolutional networks (ST-GCN) model based on skeleton features is utilized to recognize the assembly action to reduce unnecessary redundant information. Considering the disorder and occlusion of assembly parts, an improved YOLOX model is proposed to improve the focusing capability of network structure on the assembly parts that are difficult to recognize. Afterwards, taking decelerator assembly tasks as an example, a rule-based reasoning method that contains the recognition information of assembly actions and assembly parts is designed to recognize the current assembly intention. Finally, the feasibility and effectiveness of the proposed approach for recognizing human intentions are verified. The integration of assembly action recognition and assembly part recognition can facilitate the accurate operator's intention recognition in the complex and flexible HRC assembly environment.     

Operations management issues in design and control of hybrid human-robot collaborative manufacturing systems: a survey

A manufacturing system able to perform a high variety of tasks requires different types of resources. Fully automated systems using robots possess high speed, accuracy, tirelessness, and force, but they are expensive. On the other hand, human workers are intelligent, creative, flexible, and able to work with different tools in different situations. A combination of these resources forms a human-machine/robot (hybrid) system, where humans and robots perform a variety of tasks (manual, automated, and hybrid tasks) in a shared workspace. Contrarily to the existing surveys, this study is dedicated to operations management problems (focusing on the applications and features) for human and machine/robot collaborative systems in manufacturing. This research is divided into two types of interactions between human and automated components in manufacturing and assembly systems: dual resource constrained (DRC) and human-robot collaboration (HRC) optimization problems. Moreover, different characteristics of the workforce and machines/robots such as heterogeneity, homogeneity, ergonomics, and flexibility are introduced. Finally, this paper identifies the optimization challenges and problems for hybrid systems. The existing literature on HRC focuses mainly on the robotic point of view and not on the operations management and optimization aspects. Therefore, the future research directions include the design of models and methods to optimize HRC systems in terms of ergonomics, safety, and throughput. In addition, studying flexibility and reconfigurability in hybrid systems is one of the main research avenues for future research.     

基于集员估计的室内移动机器人多传感器融合定位

本文针对室内移动机器人的长距离实时鲁棒定位问题进行了研究,考虑到单一定位手段存在的不足,以二维扫描激光和里程计作为主要的定位设备,采用多传感器数据融合技术实现了移动机器人的精确定位.论文首先通过引入基于点-直线特征匹配的改进迭代最近邻(iterative closest point,ICP)扫描匹配方法对激光采集的环境点云信息进行迭代匹配以得到相对位姿变换估计,并推导了其估计不确定性的保守包络矩阵形式,然后通过建立定位过程和观测模型,引入扩展非线性集员滤波器作为多传感器融合方法,利用扫描匹配结果校正由里程计滑移带来的定位误差,并获取定位自身的不确定性边界估计.实验结果表明了本文所提出的室内定位方法的精度、实时性和鲁棒性.     

Improving human robot collaboration through Force/Torque based learning for object manipulation

Human–Robot Collaboration (HRC) is a term used to describe tasks in which robots and humans work together to achieve a goal. Unlike traditional industrial robots, collaborative robots need to be adaptive; able to alter their approach to better suit the situation and the needs of the human partner. As traditional programming techniques can struggle with the complexity required, an emerging approach is to learn a skill by observing human demonstration and imitating the motions; commonly known as Learning from Demonstration (LfD). In this work, we present a LfD methodology that combines an ensemble machine learning algorithm (i.e. Random Forest (RF)) with stochastic regression, using haptic information captured from human demonstration. The capabilities of the proposed method are evaluated using two collaborative tasks; co-manipulation of an object (where the human provides the guidance but the robot handles the objects weight) and collaborative assembly of simple interlocking parts. The proposed method is shown to be capable of imitation learning; interpreting human actions and producing equivalent robot motion across a diverse range of initial and final conditions. After verifying that ensemble machine learning can be utilised for real robotics problems, we propose a further extension utilising Weighted Random Forest (WRF) that attaches weights to each tree based on its performance. It is then shown that the WRF approach outperforms RF in HRC tasks.     

面向地下空间探测的移动机器人定位与感知方法

提出了一种面向地下空间探测的移动机器人定位与感知方法。首先,针对地下空间的结构退化问题,构建了基于因子图的激光雷达／里程计／惯性测量单元紧耦合融合框架;推导了高精度惯性测量单元／里程计的预积分模型,利用因子图算法实现对移动机器人运动状态及传感器参数的同步估计。同时,提出了基于激光雷达／红外相机融合的目标识别方法,能够对弱光照环境下的多种目标进行识别与相对定位。试验结果表明,在结构退化环境中,本文方法能够将移动机器人的定位精度提升50%以上,并对弱光照环境中的目标实现厘米级的相对定位精度。     

Event based distributed kalman filter for limited resource multirobot cooperative localization

This paper presents a multirobot cooperative event based localization scheme with improved bandwidth usage in a heterogeneous group of mobile robots. The proposed method relies on an agent based framework that defines the communications between robots and on an event based Extended Kalman Filter that performs the cooperative sensor fusion from local, global and relative sources. The event is generated when the pose error covariance exceeds a predefined limit. By this, the robots update the pose using the relative information available only when necessary, using less bandwidth and computational resources when compared to the time based methods, allowing bandwidth allocation for other tasks while extending battery life. The method is tested using a simulation platform developed in the programming language JAVA with a group of differential mobile robots represented by an agent in a JADE framework. The pose estimation performance, error covariance and number of messages exchanged in the communication are measured and used to compare the traditional time based approach with the proposed event based algorithm. Also, the compromise between the accuracy of the localization method and the bandwidth usage is analyzed for different event limits. A final experimental test with two SUMMIT XL robots is shown to validate the simulation results.     

多约束条件下机器人柔性装配技能自学习

机器人的装配策略受装配对象特性、装配工艺和装配控制方法的约束,针对装配过程接触阶段的位姿不确定性问题,提出一种装配姿态调整技能自学习的方法.首先描述多约束条件下的机器人装配技能问题,建立基于力/力矩、位姿、关节角度等多模信息描述的装配系统模型;然后构建融合竞争架构的机器人决策网络和策略优化网络,通过与环境的不断交互,进...     

Mobile robot navigation using grid line patterns via probabilistic measurement modeling

Mobile robots are generally equipped with proprioceptive motion sensors such as odometers and inertial sensors. These sensors are used for dead-reckoning navigation in an indoor environment where GPS is not available. However, this dead-reckoning scheme is susceptible to drift error in position and heading. This study proposes using grid line patterns which are often found on the surface of floors or ceilings in an indoor environment to obtain pose (i.e., position and orientation) fix information without additional external position information by artificial beacons or landmarks. The grid lines can provide relative pose information of a robot with respect to the grid structure and thus can be used to correct the pose estimation errors. However, grid line patterns are repetitive in nature, which leads to difficulties in estimating its configuration and structure using conventional Gaussian filtering that represent the system uncertainty using a unimodal function (e.g., Kalman filter). In this study, a probabilistic sensor model to deal with multiple hypotheses is employed and an online navigation filter is designed in the framework of particle filtering. To demonstrate the performance of the proposed approach, an experiment was performed in an indoor environment using a wheeled mobile robot, and the results are presented.     

机器人手部运动误差分析

本文首先引入三阶影响系数,进而系统地导出了串联机器人及并联机器人手部位姿误差、位姿速度误差、位姿加速度误差分析的显表达式,推导过程简明、形式简单,便于编程.最后给出了计算示例.     

基于单UWB融合里程计的多机器人相对定位方法

针对卫星信号受阻,无预设基础设施(定位基站、地标等)环境下多机器人间的相对定位问题,提出了一种基于单个超宽带(ultra-wideband, UWB)融合里程计的多机器人相对定位方法。该方法利用滑动窗口截取历史时刻的多组机器人间测距信息与里程计预测的机器人位姿,构建非线性最小二乘问题,实现机器人间的相对位姿估计;利用扩展卡尔曼滤波算法估计里程计协方差,并将其以加权的方式运用于非线性优化,抑制滑动窗口内里程计累积误差对定位结果的影响;最后,利用图优化算法融合里程计与非线性优化获得的相对位姿作进一步优化,抑制UWB测量误差影响,以获得稳定的相对定位结果。实验结果表明,在6 m×12 m的真实测试环境中,所提方法能够获得0.32 m的相对位置精度和4.16°的相对角度精度,相比于现有多机器人相对定位方案,该方法具有高精度、低成本、部署简单以及定位稳定的优点。     

基于双移动信标的多机器人编队控制算法

针对多机器人在未知环境下的编队控制问题,提出了一种基于双移动信标的多机器人编队算法.该方法在以两个移动信标机器人为领航机器人的基础之上,设计了基于超宽带测距技术的多机器人定位模型,通过摔制从机器人的位姿状态,实现多机器人编队控制,并且设计了多传感器数据融合算法,有效提高多机器人编队的精度.该方法解决了多机器人在未知环境中的编队控制问题,提高了多机器人编队控制的精度.仿真结果表明了该方法的可行性和有效性.