Control sharing in human-robot team interaction

The interaction between humans and robot teams is highly relevant in many application domains, for example in collaborative manufacturing, search and rescue, and logistics. It is well-known that humans and robots have complementary capabilities: Humans are excellent in reasoning and planning in unstructured environments, while robots are very good in performing tasks repetitively and precisely. In consequence, one of the key research questions is how to combine human and robot team decision making and task execution capabilities in order to exploit their complementary skills. From a controls perspective this question boils down to how control should be shared among them. This article surveys advances in human-robot team interaction with special attention devoted to control sharing methodologies. Additionally, aspects affecting the control sharing design, such as human behavior modeling, level of autonomy and human-machine interfaces are identified. Open problems and future research directions towards joint decision making and task execution in human-robot teams are discussed.     

协作机器人及其运动规划方法研究综述

协作机器人是一种新型工业机器人,利用物联网感知等新兴技术进行智能人机交互,实现与人类近距离协同工作以提高生产效率。因此,协作机器人近年来备受关注,成为了机器人领域最热门的研究方向之一。介绍了协作机器人的基本情况,包括主要产品、机器人本体设计以及应用案例;介绍了常见的人机协作方法,重点围绕人机协作中的高效、简单、安全三个特点,分别对协作机器人编程、安全协作方法、高效协作方法三个方面的研究成果展开讨论;总结了常见的机器人运动规划方法,分成路径规划方法和轨迹规划方法两部分介绍,并分析了各类方法的优缺点;最后,对目前协作机器人研究的发展方向进行了展望。     

一种基于三维建图和虚拟现实的人机交互系统

以提高人机共融水平为目的,以救援机器人为背景,提出并实现基于三维建图和虚拟现实(VR)技术的人机交互系统.在该系统中,救援机器人基于多线激光雷达和惯性测量单元(IMU)实时构建环境的三维点云地图,并将建图结果增量式地表示为3D-NDT地图,实时传输至操作台的虚拟现实系统中可视化;同时,操作人员利用虚拟现实系统的交互设备生成机器人的控制指令,控制机器人运动,构成一个完整的人在回路的人机交互系统.该系统在将机器人环境实时在虚拟现实中可视化的基础上,可以给操作人员以极强的沉浸感,有利于操作人员更直接地理解机器人所处环境.此外,该系统作为一种新的人机交互方式,为提高人与机器人的自然交互水平提供了新思路,对促进人机交互技术的发展具有重要意义.     

Accelerating Robot Development Through Integral Analysis of Human–Robot Interaction

The development of interactive robots is a complicated process, involving a plethora of psychological, technical, and contextual influences. To design a robot capable of operating ldquointelligentlyrdquo in everyday situations, one needs a profound understanding of human-robot interaction (HRI). We propose an approach based on integral analysis of multimodal data to pursue this understanding and support interdisciplinary research and development in the field of robotics. To adopt this approach, a software tool named interaction debugger was developed that features user-friendly navigation, browsing, searching, viewing, and annotation of data; it enables fine-grained inspection of the HRI. In four case studies, we demonstrated how our analysis approach aids the development process of interactive robots.     

Robust human position estimation in cooperative robotic cells

Robots are increasingly present in our lives, sharing the workspace and tasks with human co-workers. However, existing interfaces for human-robot interaction / cooperation (HRI/C) have limited levels of intuitiveness to use and safety is a major concern when humans and robots share the same workspace. Many times, this is due to the lack of a reliable estimation of the human pose in space which is the primary input to calculate the human-robot minimum distance (required for safety and collision avoidance) and HRI/C featuring machine learning algorithms classifying human behaviours / gestures. Each sensor type has its own characteristics resulting in problems such as occlusions (vision) and drift (inertial) when used in an isolated fashion. In this paper, it is proposed a combined system that merges the human tracking provided by a 3D vision sensor with the pose estimation provided by a set of inertial measurement units (IMUs) placed in human body limbs. The IMUs compensate the gaps in occluded areas to have tracking continuity. To mitigate the lingering effects of the IMU offset we propose a continuous online calculation of the offset value. Experimental tests were designed to simulate human motion in a human-robot collaborative environment where the robot moves away to avoid unexpected collisions with de human. Results indicate that our approach is able to capture the human’s position, for example the forearm, with a precision in the millimetre range and robustness to occlusions.     

医院消杀机器人作业安全与交互设计策略

作业安全与设计质量对提高医院公共环境消杀机器人"人机系统"运行效率具有关键作用.以人因工程学、交互科学、认知科学理论为视角,探究医院消杀机器人作业安全与交互设计策略.采用层次任务与表格任务结合H TA-T分析法,构建消杀作业任务流程,剖析作业流程中典型安全事故及其致因逻辑,基于工作域分析法(WDA)对消杀作业约束条件进...     

Human motion prediction for human-robot collaboration

In human-robot collaborative manufacturing, industrial robots would work alongside human workers who jointly perform the assigned tasks seamlessly. A human-robot collaborative manufacturing system is more customised and flexible than conventional manufacturing systems. In the area of assembly, a practical human-robot collaborative assembly system should be able to predict a human worker’s intention and assist human during assembly operations. In response to the requirement, this research proposes a new human-robot collaborative system design. The primary focus of the paper is to model product assembly tasks as a sequence of human motions. Existing human motion recognition techniques are applied to recognise the human motions. Hidden Markov model is used in the motion sequence to generate a motion transition probability matrix. Based on the result, human motion prediction becomes possible. The predicted human motions are evaluated and applied in task-level human-robot collaborative assembly.     

Operations management issues in design and control of hybrid human-robot collaborative manufacturing systems: a survey

A manufacturing system able to perform a high variety of tasks requires different types of resources. Fully automated systems using robots possess high speed, accuracy, tirelessness, and force, but they are expensive. On the other hand, human workers are intelligent, creative, flexible, and able to work with different tools in different situations. A combination of these resources forms a human-machine/robot (hybrid) system, where humans and robots perform a variety of tasks (manual, automated, and hybrid tasks) in a shared workspace. Contrarily to the existing surveys, this study is dedicated to operations management problems (focusing on the applications and features) for human and machine/robot collaborative systems in manufacturing. This research is divided into two types of interactions between human and automated components in manufacturing and assembly systems: dual resource constrained (DRC) and human-robot collaboration (HRC) optimization problems. Moreover, different characteristics of the workforce and machines/robots such as heterogeneity, homogeneity, ergonomics, and flexibility are introduced. Finally, this paper identifies the optimization challenges and problems for hybrid systems. The existing literature on HRC focuses mainly on the robotic point of view and not on the operations management and optimization aspects. Therefore, the future research directions include the design of models and methods to optimize HRC systems in terms of ergonomics, safety, and throughput. In addition, studying flexibility and reconfigurability in hybrid systems is one of the main research avenues for future research.     

基于认知模型的室内移动服务机器人人机耦合协同作业机制

针对老年人和残疾人这类特殊用户群体与服务机器人构成的人机智能系统,提出了基于ACT-R(理性思维的适应性控制)认知架构模型的室内移动服务机器人人机耦合协同作业机制.基于ACT-R认知架构对人机一体化室内移动服务机器人人机协同作业系统进行了总体设计,利用简单自然的人机效应通道,设计了基于ACT-R认知架构的人机耦合界面;通过人-机-环境空间感知耦合,提出并建立了室内移动服务机器人人机一体化协同决策作业机制.最后在室内环境下进行移动服务机器人人机协同作业实验,系统安全高效地完成了作业任务,验证了该机制的有效性.     

Trust as indicator of robot functional and social acceptance. An experimental study on user conformation to iCub answers

To investigate the dynamics of human-robot acceptance, we carried out an experimental study with 56 adult participants and the iCub robot. Trust in the robot has been considered as a main indicator of acceptance and measured by the participants' conformation to the iCub's answers to questions on functional and social tasks characterized by perceptual and socio cognitive uncertainty. In particular, we were interested in understanding whether (i) trust in functional savvy is a prerequisite for trust in social savvy, and (ii) to what extent factors such as participants' desire for control, attitude towards social influence of robots, and imagined collaborative vs. competitive scenario, may influence their trust in the iCub. We found that participants conformed more to the iCub's answers in the functional than in the social tasks. Moreover, the few participants conforming to the iCub's answers in the social task also conformed less in the functional issues. Trust in the robot's functional savvy does not thus seem to be a pre-requisite for trust in its social savvy. Finally, the examined factors did not influence the trust in iCub. Results are discussed with relation to methodology of human-robot interaction (HRI) research.     

An AR-assisted Deep Reinforcement Learning-based approach towards mutual-cognitive safe human-robot interaction

With the emergence of Industry 5.0, the human-centric manufacturing paradigm requires manufacturing equipment (robots, etc.) interactively assist human workers to deal with dynamic and complex production tasks. To achieve symbiotic human–robot interaction (HRI), the safety issue serves as a prerequisite foundation. Regarding the growing individualized demand of manufacturing tasks, the conventional rule-based safe HRI measures could not well address the safety requirements due to inflexibility and lacking synergy. To fill the gap, this work proposes a mutual-cognitive safe HRI approach including worker visual augmentation, robot velocity control, Digital Twin-enabled motion preview and collision detection, and Deep Reinforcement Learning-based robot collision avoidance motion planning in the Augmented Reality-assisted manner. Finally, the feasibility of the system design and the performance of the proposed approach are validated by establishing and executing the prototype HRI system in a practical scene.     

Analysis of quality standards for industrial collaborative robots based on user-centered design framework

Industrial collaborative robots have become increasingly important in recent years due to their ability to work safely and efficiently alongside humans. As a result, there is a growing need for evaluation standards to ensure the quality of collaborative robots. However, existing studies only consider system-centered and technical aspects of collaborative robots, and there is a lack of research on user-centered quality evaluation. In this study, we identified 21 user requirements based on a user-centered design framework and confirmed the limitations of existing quality standards by reviewing the standard clauses for collaborative robots. It was found that user needs to be related to performance, safety, and even usability and enjoyment are already being expressed according to the user-centered design framework, but the quality standards for these needs only present design principles or do not consider them at all. This study provides information on the quality attributes that need to be fulfilled to satisfy user requirements and suggests the need and direction for further research on the user-centered evaluation of collaborative robots. Accordingly, the user's perception and experience of collaborative robots are expected to improve.     

Human–robot interaction in industrial collaborative robotics: a literature review of the decade 2008–2017

ABSTRACT

Currently, a large number of industrial robots have been deployed to replace or assist humans to perform various repetitive and dangerous manufacturing tasks. However, based on current technological capabilities, such robotics field is rapidly evolving so that humans are not only sharing the same workspace with robots, but also are using robots as useful assistants. Consequently, due to this new type of emerging robotic systems, industrial collaborative robots or cobots, human and robot co-workers have been able to work side-by-side as collaborators to accomplish tasks in industrial environments. Therefore, new human–robot interaction systems have been developed for such systems to be able to utilize the capabilities of both humans and robots. Accordingly, this article presents a literature review of major recent works on human–robot interactions in industrial collaborative robots, conducted during the last decade (between 2008 and 2017). Additionally, the article proposes a tentative classification of the content of these works into several categories and sub-categories. Finally, this paper addresses some challenges of industrial collaborative robotics and explores future research issues.     

Evaluating the Robot Personality and Verbal Behavior of Domestic Robots Using Video-Based Studies

Robots are increasingly being used in domestic environments and should be able to interact with inexperienced users. Human–human interaction and human–computer interaction research findings are relevant, but often limited because robots are different from both humans and computers. Therefore, new human–robot interaction (HRI) research methods can inform the design of robots suitable for inexperienced users. A video-based HRI (VHRI) methodology was here used to carry out a multi-national HRI user study for the prototype domestic robot BIRON (BIelefeld RObot companioN). Previously, the VHRI methodology was used in constrained HRI situations, while in this study HRIs involved a series of events as part of a 'hometour' scenario. Thus, the present work is the first study of this methodology in extended HRI contexts with a multi-national approach. Participants watched videos of the robot interacting with a human actor and rated two robot behaviors (Extrovert and Introvert). Participants' perceptions and ratings of the robot's behaviors differed with regard to both verbal interactions and person following by the robot. The study also confirms that the VHRI methodology provides a valuable means to obtain early user feedback, even before fully working prototypes are available. This can usefully guide the future design work on robots, and associated verbal and non-verbal behaviors.     

Autonomy and Common Ground in Human-Robot Interaction: A Field Study

The use of robots, especially autonomous mobile robots, to support work is expected to increase over the next few decades. However, little empirical research examines how users form mental models of robots, how they collaborate with them, and what factors contribute to the success or failure of human-robot collaboration. A two-year observational study of a collaborative human-robot system suggests that the factors disrupting the creation of common ground for interactive communication change at different levels of robot autonomy. Our observations of users collaborating with the remote robot showed differences in how the users reached common ground with the robot in terms of an accurate, shared understanding of the robot's context, planning, and actions - a process called grounding. We focus on how the types and levels of robot autonomy affect grounding. We also examine the challenges a highly autonomous system presents to people's ability to maintain a shared mental model of the robot     

Designing LED lights for a robot to communicate gaze

ABSTRACT

Eye gaze is considered to be a particularly important non-verbal communication cue. Gaze research is also becoming a hot topic in human–robot interaction (HRI). However, research on social eye gaze for HRI focuses mainly on human-like robots. There remains a lack of methods for functional robots, which are constrained in appearance, to show gaze-like behavior. In this work, we investigate how we can implement gaze behavior in functional robots to assist humans in reading their intent. We explore design implications based on LED lights as we consider LEDs to be easily installed in most robots while not introducing features that are too human-like (to prevent users from having high expectations towards the robots). In this paper, we first developed a design interface that allows designers to freely test different parameter settings for an LED-based gaze display for a Roomba robot. We summarized design principles for well simulating LED-based gazes. Our suggested design is further evaluated by a large group of participants with regard to their perception and interpretation of the robot's behaviors. On the basis of the findings, we finally offer a set of design implications that can be beneficial to HRI and HCI researchers.     

Hybrid machine learning for human action recognition and prediction in assembly

As one of the critical elements for smart manufacturing, human-robot collaboration (HRC), which refers to goal-oriented joint activities of humans and collaborative robots in a shared workspace, has gained increasing attention in recent years. HRC is envisioned to break the traditional barrier that separates human workers from robots and greatly improve operational flexibility and productivity. To realize HRC, a robot needs to recognize and predict human actions in order to provide assistance in a safe and collaborative manner. This paper presents a hybrid approach to context-aware human action recognition and prediction, based on the integration of a convolutional neural network (CNN) and variable-length Markov modeling (VMM). Specifically, a bi-stream CNN structure parses human and object information embedded in video images as the spatial context for action recognition and collaboration context identification. The dependencies embedded in the action sequences are subsequently analyzed by a VMM, which adaptively determines the optimal number of current and past actions that need to be considered in order to maximize the probability of accurate future action prediction. The effectiveness of the developed method is evaluated experimentally on a testbed which simulates an assembly environment. High accuracy in both action recognition and prediction is demonstrated.     

满足不同交互任务的人机共融系统设计

人与机器人共同协作的灵活生产模式已经成为工业成产的迫切需求,因此,近年来人机共融系统方面的研究受到了越来越多关注.设计并实现了一种满足不同交互任务的人机共融系统,人体动作的估计和机器人的交互控制是其中的关键技术.首先,提出了一种基于多相机和惯性测量单元信息融合的人体姿态解算方法,通过构造优化问题,融合多相机下的2D关节检测信息和所佩戴的惯性测量单元测量信息,对人体运动学姿态进行优化估计,改善了单一传感器下,姿态信息不全面以及对噪声敏感的问题,提升了姿态估计的准确度.其次,结合机器人的运动学特性和人机交互的特点,设计了基于目标点跟踪和模型预测控制的机器人控制策略,使得机器人能够通过调整控制参数,适应动态的环境和不同的交互需求,同时保证机器人和操作人员的安全.最后,进行了动作跟随、物品传递、主动避障等人机交互实验,实验结果表明了所设计的机器人交互系统在人机共融环境下的有效性和可靠性.