Difficulty and complexity definitions for assembly task allocation and assignment in human–robot collaborations: A review

This paper presents a literature review on the different aspects of task allocation and assignment problems in human–robot collaboration (HRC) tasks in industrial assembly environments. In future advanced industrial environments, robots and humans are expected to share the same workspace and collaborate to efficiently achieve shared goals. Difficulty- and complexity-aware HRC assembly is necessary for human-centric manufacturing, which is a goal of Industry 5.0. Therefore, the objective of this study is to clarify the definitions of difficulty and complexity used to encourage effective collaboration between humans and robots to leverage the adaptability of humans and the autonomy of robots. To achieve this goal, a systematic review of the following relevant databases for computer science was performed: IEEE Xplore, ScienceDirect, SpringerLink, ACM Digital Library, and ASME Digital Collection. The results extracted from 74 peer-reviewed research articles published until July 2022 were summarized and categorized into four taxonomies for 145 difficulty and complexity definitions from the perspectives of (1) definition-use objectives, (2) evaluation objectives, (3) evaluation factors, and (4) evaluation variables. Next, existing definitions were primarily classified according to the following two criteria to identify potential future studies on the formulation of new definitions for human-centric manufacturing: (1) agent specificity and (2) common aspects in manual and robotic assemblies.     

Neural reactive path planning with Riemannian motion policies for robotic silicone sealing

Due to its excellent chemical and mechanical properties, silicone sealing has been widely used in many industries. Currently, the majority of these sealing tasks are performed by human workers. Hence, they are susceptible to labor shortage problems. The use of vision-guided robotic systems is a feasible alternative to automate these types of repetitive and tedious manipulation tasks. In this paper, we present the development of a new method to automate silicone sealing with robotic manipulators. To this end, we propose a novel neural path planning framework that leverages fractional-order differentiation for robust seam detection with vision and a Riemannian motion policy for effectively learning the manipulation of a sealing gun. Optimal control commands can be computed analytically by designing a deep neural network that predicts the acceleration and associated Riemannian metric of the sealing gun from feedback signals. The performance of our new methodology is experimentally validated with a robotic platform conducting multiple silicone sealing tasks in unstructured situations. The reported results demonstrate that compared with directly predicting the control commands, our neural path planner achieves a more generalizable performance on unseen workpieces and is more robust to human/environment disturbances.     

Robust techniques for abandoned and removed object detection based on Markov random field

This paper presents a novel framework for detecting abandoned objects by introducing a fully-automatic GrabCut object segmentation. GrabCut seed initialization is treated as a background (BG) modelling problem that focuses only on unhanded objects and objects that become immobile. The BG distribution is constructed with dual Gaussian mixtures that are comprised of high and low learning rate models. We propose a primitive BG model-based removed object validation and Haar feature-based cascade classifier for still-people detection once a candidate for a released object has been detected. Our system can obtain more robust and accurate results for real environments based on evaluations of realistic scenes from CAVIAR, PETS2006, CDnet 2014, and our own datasets.     

Novel magnetic field sensor based on magnetic fluids infiltrated dual-core photonic crystal fibers

Novel magnetic field sensor based on magnetic fluids infiltrated dual-core Photonic Crystal Fibers (PCFs) is proposed in this paper. Inside the cross-section of the designed PCFs, the two fiber cores filled with magnetic fluids (Fe₃O₄) are separated by an air hole, and then form two independent waveguides with mode coupling. The mode coupling under different magnetic field strength is investigated theoretically. A novel and simple magnetic field sensing system is proposed and its sensing performances have been studied numerically. The results show that the magnetic field sensor with 15-cm PCFs has a large sensing range and high sensitivity of 4.80 pm/Oe. It provides a new feasible method to design PCF-based magnetic field sensor.     

Application of facial reduction to H ∞ state feedback control problem

One often encounters numerical difficulties in solving linear matrix inequality (LMI) problems obtained from H _∞ control problems. For semidefinite programming (SDP) relaxations for combinatorial problems, it is known that when either an SDP relaxation problem or its dual is not strongly feasible, one may encounter such numerical difficulties. We discuss necessary and sufficient conditions to be not strongly feasible for an LMI problem obtained from H _∞ state feedback control problems and its dual. Moreover, we interpret the conditions in terms of control theory. In this analysis, facial reduction, which was proposed by Borwein and Wolkowicz, plays an important role. We show that the dual of the LMI problem is not strongly feasible if and only if there exist invariant zeros in the closed left-half plane in the system, and present a remedy to remove the numerical difficulty with the null vectors associated with invariant zeros in the closed left-half plane. Numerical results show that the numerical stability is improved by applying it.     

改进SSD的输电铁塔鸟窝检测

输电铁塔作为架空输电线的重要组成部分,其安全状况将影响整个电力系统的运行,鸟窝的搭建作为影响输电线路正常运行的重要因素之一,需要对此进行监控.而现有的监控手段不仅效率低,还需要耗费大量的人力物力.本文针对这一现象提出一种基于SSD算法的实时检测方法,并在SSD的网络结构基础上将前置网络VGGNet替换为ResNet-101,提高其特征提取能力,并将Softmax loss用Focal loss代替以改善SSD算法中样本不平衡的问题,并利用数据增广提高数据多样性,以提高模型的鲁棒性.实验结果表明本文所提出的方法检测精度对比原SSD算法在准确度和召回率上分别有3.17%和6.35%的提升.     

Adaptive hierarchical positioning error compensation for long-term service of industrial robots based on incremental learning with fixed-length memory window and incremental model reconstruction

Industrial robots have been extensively used in industry, however, geometric errors mainly caused by connecting rod parameter error and non-geometric errors caused by deflection and friction, etc., limit its application in high-accuracy machining. Aiming at addressing these two types of errors, parametric methods for error compensation based on the kinematic model and non-parametric methods of directly establishing the mapping relationship between the actual and target poses of the robot end-effector are investigated and proposed. Currently both types of methods are mainly offline and will be no longer applicable when the pose of the end-effector in the workspace changes dramatically or the working performance of the robot degrades. Thus, to compensate the positioning error of an industrial robot during long-term operation, this research proposes an adaptive hierarchical compensation method based on fixed-length memory window incremental learning and incremental model reconstruction. Firstly, the correlation between positioning errors and robot poses is studied, a calibration sample library is created, and thus the actively evaluating mechanism of the pose mapping model is established to overcome the problem of the robot’ workspace having a differential distribution of error levels. Then, an incremental learning algorithm with fixed-length memory window and an incremental model reconstruction algorithm are designed to optimize the pose mapping model in terms of its parameters and architecture and overcome the problem that the performance degradation of the robot exacerbates the positioning error and affects the applicability of the pose mapping model, ensuring that the pose mapping model runs stably above the target accuracy level. Finally, the proposed method is applied to the long-term compensation case of a Stäubli industrial robot and a UR robot, and compared to state-of-art methods. Verification results show the proposed method reduces the position error of the Stäubli robot from 0.85mm to 0.13mm and orientation error from 0.68° to 0.07°, as well as reduces the position error of the UR robot from 2.11mm to 0.17mm, demonstrating that the proposed method works in real world scenarios and outperforms similar methods.