Variable admittance force feedback device and its human-robot interaction stability

In teleoperation, a force feedback device is a medium to build a transparent interaction environment between a human and a remote robotic arm. Using force feedback devices, the users can operate the remote robotic arm intuitively and perceive remote interaction through the force channel, just as if they are in the remote environment. Compared with impedance devices, admittance devices have the advantages of large feedback output, high stiffness, high reverse driving performance, and flexible structure, which are more suitable for the teleoperation of heavy-duty and large-size robotic arms. However, the control of admittance devices is relatively complex and has some inherent limitations such as response delay, instability from high-frequency oscillation, difficulty in achieving constant speed control, etc. Errors in admittance model parameters and human physiological characteristics, such as force application fluctuations, are the root causes of these problems. In this study, we proposed a fuzzy variable damping admittance algorithm, which allows the device to identify the user's movement intention and give respond quickly and accurately. We also established a human-robot interaction (HRI) system model of an admittance master controller device and summarized the principles of the admittance parameter configuration of a stable system. For the device's high-frequency oscillation instability caused by human arm stiffness, we propose an oscillation observation and reduction algorithm. By observing the force signal change characteristics, the algorithm can quickly detect the unstable behavior caused by human hands and perform oscillation reduction. To reduce the influence on upper limb uniform motion caused by fluctuating force application, we proposed a constant velocity intention inference algorithm based on a velocity spherical cone to smooth out the device operating velocity to achieve smooth control. The method proposed in this study achieved stable control in a 6 DOF force feedback device as a master controller, and the effect has been verified by experiments.     

Controller design for human-robot interaction

Many robotics tasks require a robot to share the same workspace with humans. In such settings, it is important that the robot performs in such a way that does not cause distress to humans in the workspace. In this paper, we address the problem of designing robot controllers which minimize the stress caused by the robot while performing a given task. We present a novel, data-driven algorithm which computes human-friendly trajectories. The algorithm utilizes biofeedback measurements and combines a set of geometric controllers to achieve human friendliness. We evaluate the comfort level of the human using a Galvanic Skin Response (GSR) sensor. We present results from a human tracking task, in which the robot is required to stay within a specified distance without causing high stress values.

AR-based interaction for human-robot collaborative manufacturing

Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. This paper proposes a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup and a wearable AR gear (HoloLens). The workspace model and UIs are evaluated in a realistic diesel engine assembly task. The AR-based interactive UIs provide 21–24% and 57–64% reduction in the task completion and robot idle time, respectively, as compared to a baseline without interaction and workspace sharing. However, user experience assessment reveal that HoloLens based AR is not yet suitable for industrial manufacturing while the projector-mirror setup shows clear improvements in safety and work ergonomics.     

Pre-collision safety strategies for human-robot interaction

Safe planning and control is essential to bringing human-robot interaction into common experience. This paper presents an integrated human−robot interaction strategy that ensures the safety of the human participant through a coordinated suite of safety strategies that are selected and implemented to anticipate and respond to varying time horizons for potential hazards and varying expected levels of interaction with the user. The proposed planning and control strategies are based on explicit measures of danger during interaction. The level of danger is estimated based on factors influencing the impact force during a human-robot collision, such as the effective robot inertia, the relative velocity and the distance between the robot and the human. A second key requirement for improving safety is the ability of the robot to perceive its environment, and more specifically, human behavior and reaction to robot movements. This paper also proposes and demonstrates the use of human monitoring information based on vision and physiological sensors to further improve the safety of the human robot interaction. A methodology for integrating sensor-based information about the user's position and physiological reaction to the robot into medium and short-term safety strategies is presented. This methodology is verified through a series of experimental test cases where a human and an articulated robot respond to each other based on the human's physical and physiological behavior.

人机交互中的个性化情感模型

人与机器人的交互过程中,情感因素的引入能够使人机交流更加自然和谐.因此,完整的人工情感模型的建立是首要解决的问题.基于情感能量理论基础,首先,提出了心境自发转移和刺激转移模型.其次,结合情绪自发转移的马尔可夫链模型和刺激转移的HMM模型,将心境和情绪的自发和刺激转移过程统一在一个框架下.最后,将完整的人工情感模型软件化并应用于儿童玩伴机器人上,在接受非结构化环境与用户的信息输入后,个性化的情感软件模块产生输出,实现针对儿童用户的玩伴机器人个性化交互,通过应用验证了该模型的有效性.     

Control sharing in human-robot team interaction

The interaction between humans and robot teams is highly relevant in many application domains, for example in collaborative manufacturing, search and rescue, and logistics. It is well-known that humans and robots have complementary capabilities: Humans are excellent in reasoning and planning in unstructured environments, while robots are very good in performing tasks repetitively and precisely. In consequence, one of the key research questions is how to combine human and robot team decision making and task execution capabilities in order to exploit their complementary skills. From a controls perspective this question boils down to how control should be shared among them. This article surveys advances in human-robot team interaction with special attention devoted to control sharing methodologies. Additionally, aspects affecting the control sharing design, such as human behavior modeling, level of autonomy and human-machine interfaces are identified. Open problems and future research directions towards joint decision making and task execution in human-robot teams are discussed.     

A population randomization-based multi-objective genetic algorithm for gesture adaptation in human-robot interaction

In recent years,vision-based gesture adaptation has attracted great attention from many experts in the field of human-robot interaction,and many methods have be...     

Digital twins for collaborative robots: A case study in human-robot interaction

Human-robot collaboration (HRC) can expand the level of automation in areas that have conventionally been difficult to automate such as assembly. However, the need of adaptability and the dynamics of human presence are keeping the full potential of human-robot collaborative systems difficult to achieve. This paper explores the opportunities of using a digital twin to address the complexity of collaborative production systems through an industrial case and a demonstrator. A digital twin, as a virtual counterpart of a physical human-robot assembly system, is built as a ‘front-runner’ for validation and control throughout its design, build and operation. The forms of digital twins along system's life cycle, its building blocks and the potential advantages are presented and discussed. Recommendations for future research and practice in the use of digital twins in the field of cobotics are given.     

Facial expression recognition and tracking for intelligent human-robot interaction

For effective interaction between humans and socially adept, intelligent service robots, a key capability required by this class of sociable robots is the successful interpretation of visual data. In addition to crucial techniques like human face detection and recognition, an important next step for enabling intelligence and empathy within social robots is that of emotion recognition. In this paper, an automated and interactive computer vision system is investigated for human facial expression recognition and tracking based on the facial structure features and movement information. Twenty facial features are adopted since they are more informative and prominent for reducing the ambiguity during classification. An unsupervised learning algorithm, distributed locally linear embedding (DLLE), is introduced to recover the inherent properties of scattered data lying on a manifold embedded in high-dimensional input facial images. The selected person-dependent facial expression images in a video are classified using the DLLE. In addition, facial expression motion energy is introduced to describe the facial muscle’s tension during the expressions for person-independent tracking for person-independent recognition. This method takes advantage of the optical flow which tracks the feature points’ movement information. Finally, experimental results show that our approach is able to separate different expressions successfully.     

Model reference adaptive impedance control for physical human-robot interaction

This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.     

实现人身和机器人交流的神经振动子控制算法

为实现人身和机器人交流时的运动同步,首先提出了一类非线性多关节神经振动子运动控制算法,其输入为机器人和人相互作用所产生的关节扭矩信号,输出为机器人关节期望角度;然后对具有代表性的二关节神经振动子控制算法中各参数的耦合特性进行了分析;最后,基于7自由度机器人臂平台对该神经振动子控制算法的有效性进行实验.实验结果表明,该控制算法能够实现人和机器人相互运动的同步,通过调节神经振动子的增益参数,同步的程度能够被改变.     

A method of vertical and horizontal force estimation by using air-filled material and camera for soft physical human-robot interaction: fundamental experiments

A method of vertical and horizontal force estimation for soft physical human–robot interaction by using an air-filled material and a camera was proposed in this study. First, we considered hypotheses for vertical and horizontal force applied to the air-filled material. Second, the prototype of air-filled material and vision-based force-sensing device (AVFSD) was constructed. Based on the result of preliminary experiments, the relationships between force, pneumatic pressure, and contact surface displacement were formulated. The vertical and horizontal force could be estimated using the AVFSD with percentage errors of 3.6 and 5.8%, respectively, and the hypotheses were verified through the experiments.     

基于人机社会力模型的人群疏散算法

针对公共场合紧急情况下人群疏散困难和效果有限的问题,提出一种基于人机社会力模型的机器人疏散人群的方法。首先,基于原始社会力模型提出了一种新的人机社会力模型,该模型在原始社会力模型的基础上加入了机器人对人作用的人机作用力;然后,基于人机社会力模型提出一种新的利用机器人疏散人群的方法,该方法在人群疏散场景中加入运动机器人,通过机器人自身的运动,利用人机作用力影响周围行人的运动状态,减小行人之间的压力,从而达到加快人群运动速度、提高人群疏散效率的目的。在室内封闭场景人群逃生、两群行人交错这两种典型的疏散场景中分别进行仿真实验,并将实验结果与未加入机器人的人群疏散结果进行对比分析,实验结果表明,基于人机社会力模型的机器人疏散人群的方法能够明显加快人群的运动,提高人群的疏散效率。     

Validating human-robot interaction schemes in multitasking environments

The ability of robots to autonomously perform tasks is increasing. More autonomy in robots means that the human managing the robot may have available free time. It is desirable to use this free time productively, and a current trend is to use this available free time to manage multiple robots. We present the notion of neglect tolerance as a means for determining how robot autonomy and interface design determine how free time can be used to support multitasking, in general, and multirobot teams, in particular. We use neglect tolerance to 1) identify the maximum number of robots that can be managed; 2) identify feasible configurations of multirobot teams; and 3) predict performance of multirobot teams under certain independence assumptions. We present a measurement methodology, based on a secondary task paradigm, for obtaining neglect tolerance values that allow a human to balance workload with robot performance.     

A systems approach to task allocation of human-robot interaction in manufacturing

Robots are sharing the manufacturing work environment with humans at an ever increasing rate. As such it has become imperative to analyze the manufacturing tasks and allocate them properly between humans and robots. A systems approach to task allocation has been taken in this paper that includes inventory of anticipated common tasks in manufacturing, design of products to be manufactured, allocation of tasks between humans and robots, and iterative improvement in product design.     

一种基于三维建图和虚拟现实的人机交互系统

以提高人机共融水平为目的,以救援机器人为背景,提出并实现基于三维建图和虚拟现实(VR)技术的人机交互系统.在该系统中,救援机器人基于多线激光雷达和惯性测量单元(IMU)实时构建环境的三维点云地图,并将建图结果增量式地表示为3D-NDT地图,实时传输至操作台的虚拟现实系统中可视化;同时,操作人员利用虚拟现实系统的交互设备生成机器人的控制指令,控制机器人运动,构成一个完整的人在回路的人机交互系统.该系统在将机器人环境实时在虚拟现实中可视化的基础上,可以给操作人员以极强的沉浸感,有利于操作人员更直接地理解机器人所处环境.此外,该系统作为一种新的人机交互方式,为提高人与机器人的自然交互水平提供了新思路,对促进人机交互技术的发展具有重要意义.     

Enabling effective human-robot interaction using perspective-taking in robots

We propose that an important aspect of human-robot interaction is perspective-taking. We show how perspective-taking occurs in a naturalistic environment (astronauts working on a collaborative project) and present a cognitive architecture for performing perspective-taking called Polyscheme. Finally, we show a fully integrated system that instantiates our theoretical framework within a working robot system. Our system successfully solves a series of perspective-taking problems and uses the same frames of references that astronauts do to facilitate collaborative problem solving with a person.