Human–robot interaction in industrial collaborative robotics: a literature review of the decade 2008–2017

ABSTRACT

Currently, a large number of industrial robots have been deployed to replace or assist humans to perform various repetitive and dangerous manufacturing tasks. However, based on current technological capabilities, such robotics field is rapidly evolving so that humans are not only sharing the same workspace with robots, but also are using robots as useful assistants. Consequently, due to this new type of emerging robotic systems, industrial collaborative robots or cobots, human and robot co-workers have been able to work side-by-side as collaborators to accomplish tasks in industrial environments. Therefore, new human–robot interaction systems have been developed for such systems to be able to utilize the capabilities of both humans and robots. Accordingly, this article presents a literature review of major recent works on human–robot interactions in industrial collaborative robots, conducted during the last decade (between 2008 and 2017). Additionally, the article proposes a tentative classification of the content of these works into several categories and sub-categories. Finally, this paper addresses some challenges of industrial collaborative robotics and explores future research issues.     

Human error probability estimation by coupling simulator data and deterministic analysis

Operator error in diagnosis and execution of task have significant impact on Nuclear Power Plant (NPP) safety. These human errors are classified as mistakes (rule base and knowledge based errors), slip (skill based) and lapses (skill based). Depending on the time of occurrence, human errors have been categorized as i) Category ‘A’ (Pre-Initiators): actions during routine maintenance and testing wherein errors can cause equipment malfunction ii) Category ‘B’ (Initiators): actions contributing to initiating events or plant transients iii) Category ‘C’ (Post-Initiators): actions involved in operator response to an accident. There have been accidents in NPPs because of human error in an operator's diagnosis and execution of an event. These underline the need to appropriately estimate HEP in risk analysis. There are several methods that are being practiced in Probabilistic Safety Assessment (PSA) studies for quantification of human error probability. However, there is no consensus on a single method that should be used. In this paper a method for estimating HEP is proposed which is based on simulator data for a particular accident scenario. For accident scenarios, the data from real NPP control room is very sparsely available. In the absence of real data, simulator based data can be used. Simulator data is expected to provide a glimpse of probable human behavior in real accident situation even though simulator data is not a substitute for real data. The proposed methodology considers the variation in crew performance time in simulator exercise and in available time from deterministic analysis, and couples them through their respective probability distributions to obtain HEP. The emphasis is on suitability of the methodology rather than particulars of the cited example.     

A novel measurement of intersegmental forces exerted on human lower limbs during forward falling and regaining balance motions

This study proposes an analysis system for evaluating the intersegmental forces exerted on human lower limbs. In existing analysis systems, the ground reaction force exerted on the feet is generally measured using force plates fixed on the motion trajectory, and the intersegmental forces are then estimated using an inverse dynamics approach. However, force plates are inconvenient and expensive. Accordingly, the present study proposes a method for evaluating the intersegmental forces without the need for force plates. In the proposed approach, the supporting and striding legs are modeled as a seven-link manipulator system, and the intersegmental forces is then derived using Newton–Euler theory. It is found that the estimated results are in good agreement with the actual measurement datum. Thus, the basic validity of the proposed analysis system is confirmed. For application, the intersegmental force would be the major reference datum to design the artificial joints.     

A human visual system-driven image segmentation algorithm

This paper presents a novel image segmentation algorithm driven by human visual system (HVS) properties. Segmentation quality metrics, based on perceptual properties of HVS with respect to segmentation, are integrated into an energy function. The energy function encodes the HVS properties from both region-based and boundary-based perspectives, where the just-noticeable difference (JND) model is employed when calculating the difference between the image contents. Extensive experiments are carried out to compare the performances of three variations of the presented algorithm and several representative segmentation and clustering algorithms available in the literature. The results show superior performance of our approach.     

Human–Cyber–Physical Systems (HCPSs) in the Context of New-Generation Intelligent Manufacturing

An intelligent manufacturing system is a composite intelligent system comprising humans, cyber systems, and physical systems with the aim of achieving specific manufacturing goals at an optimized level. This kind of intelligent system is called a human–cyber–physical system (HCPS). In terms of technology, HCPSs can both reveal technological principles and form the technological architecture for intelligent manufacturing. It can be concluded that the essence of intelligent manufacturing is to design, construct, and apply HCPSs in various cases and at different levels. With advances in information technology, intelligent manufacturing has passed through the stages of digital manufacturing and digital-networked manufacturing, and is evolving toward new-generation intelligent manufacturing (NGIM). NGIM is characterized by the in-depth integration of new-generation artificial intelligence (AI) technology (i.e., enabling technology) with advanced manufacturing technology (i.e., root technology); it is the core driving force of the new industrial revolution. In this study, the evolutionary footprint of intelligent manufacturing is reviewed from the perspective of HCPSs, and the implications, characteristics, technical frame, and key technologies of HCPSs for NGIM are then discussed in depth. Finally, an outlook of the major challenges of HCPSs for NGIM is proposed.     

Using Neural Networks to predict HFACS unsafe acts from the pre-conditions of unsafe acts

Abstract

Human Factors Analysis and Classification System (HFACS) is based upon Reason’s organizational model of human error which suggests that there is a ‘one to many’ mapping of condition tokens (HFACS level 2 psychological precursors) to unsafe act tokens (HFACS level 1 error and violations). Using accident data derived from 523 military aircraft accidents, the relationship between HFACS level 2 preconditions and level 1 unsafe acts was modelled using an artificial neural network (NN). This allowed an empirical model to be developed congruent with the underlying theory of HFACS. The NN solution produced an average overall classification rate of ca. 74% for all unsafe acts from information derived from their level 2 preconditions. However, the correct classification rate was superior for decision- and skill-based errors, than for perceptual errors and violations.

Practitioner Summary: A model to predict unsafe acts (HFACS level 1) from their preconditions (HFACS level 2) was developed from the analysis of 523 military aircraft accidents using an artificial NN. The results could correctly predict approximately 74% of errors.     

Variable-dimensional vector modulation for perceptual-based DWT blind audio watermarking with adjustable payload capacity

A variable-dimensional vector modulation (VDVM) scheme is introduced to maximize the efficiency of the norm-space DWT-based blind audio watermarking technique. This flexible scheme allows the watermarking algorithm to reach a balance between robustness and capacity, while the imperceptivity is always ensured. The imperfection of applying quantization index modulation in the open-loop case has been rectified. The effectiveness of the proposed scheme is proven using the perceptual evaluation of audio quality (PEAQ) and bit error rates of recovered watermarks under various signal processing attacks. Experimental results show that the proposed VDVM scheme is comparable to other recently developed methods in robustness and imperceptivity even at a capacity as high as 301.46 bps. Such a capacity can be further doubled by halving the dimension of the involved DWT vector, while the robustness is still maintained at a satisfactory level.     

STAR-Lite: A light-weight scalable self-taught learning framework for older adults’ activity recognition

Recognizing activities for older adults is challenging as we observe a variety of activity patterns caused due to aging (e.g., limited dexterity, limb control, slower response time) or/and underlying health conditions (e.g., dementia). However, existing literature with deep learning methods has successfully recognized activities when the dataset contains high-quality annotations and is captured in a controlled environment. On the contrary, data captured in a real-world environment, especially with older adults exhibiting memory-related symptoms, varying psychological and mental health status, reliance on caregivers to perform daily activities, and unavailability of domain-specific annotators, makes obtaining quality data with annotations challenging; leaving us with limited labeled data and abundant unlabeled data. In this paper, we hypothesize that projecting the labeled data representations comprising a specific set of activities onto a new representation space characterized by the unlabeled data comprising activities beyond the limited activities in the labeled dataset would help us rely less on the annotated data to improve activity detection performance. Motivated by this, we propose STAR-Lite, a self-taught learning framework that involves a pre-training framework to prepare the new representation space considering activities beyond the initial labels in the labeled dataset. STAR-Lite projects the labeled data representations on the new representation space characterized by unlabeled data labels and learns higher-level representations of the labeled dataset while optimizing inter- and intra- class distances without explicitly using a computation hungry similarity-based approach. We demonstrate that our proposed approach, STAR-Lite (a) improves activity recognition performance in a supervised setting and (b) is feasible for real-world deployment. To enhance the feasibility of deploying STAR-Lite on devices with limited memory resources, we explore model compression techniques such as pruning and quantization and propose a novel layer-wise pruning-rate optimization technique that effectively compresses the network while preserving the model performance. The evaluation was performed using the Alzheimer’s Activity Recognition dataset (AAR) captured from 25 individuals living in a retirement community center with IRB approval (#Y18NR12035) using an in-house SenseBox infrastructure while concurrently assessing the clinical evaluation of the participants for dementia, and independent living. Our extensive evaluation reveals that STAR-Lite can detect activities with an F1-score of 85.12% despite 62% reduction in model size and 5% improvement of execution time on a resource constrained device.     

Basic human–robot interaction system running on an embedded platform

Robotics will be a dominant area in society throughout future generations. Its presence is currently increasing in most daily life settings, with devices and mechanisms that facilitate the accomplishment of diverse tasks, as well as in work scenarios, where machines perform more and more jobs. This increase in the presence of autonomous robotic systems in society is due to their great efficiency and security compared to human capacity, which is thanks mainly to the enormous precision of their sensor and actuator systems. Among these, vision sensors are of the utmost importance. Humans and many animals naturally enjoy powerful perception systems, but, in robotics, this constitutes a constant line of research. In addition to having a high capacity for reasoning and decision-making, these robots incorporate important advances in their perceptual systems, allowing them to interact effectively in the working environments of this new industrial revolution. Drawing on the most basic interaction between humans, looking at the face, an innovative system is presented in this paper, which was developed for an autonomous and DIY robot. This system is composed of three modules. First, the face detection component, which detects human faces in the current image. Second, the scene representation algorithm, which offers a wider field of view than that of the single camera used, mounted on a servo-pan unit. Third, the active memory component, which was designed and implemented according to two competing dynamics: life and salience. The algorithm intelligently moves the servo-pan unit with the aim of finding new faces, follow existing ones and forgetting those that no longer appear on the scene. The system was developed and validated using a low-cost platform based on a Raspberry Pi3 board.