一类具有相同结构的表情机器人共同注意方法

为了提高使用者与机器人人机合作过程中的社交和认知能力,针对一类具有相同结构的表情机器人,提出了一种基于静态视觉的人机共同注意实现方法.首先,获取使用者注意焦点的影像像素坐标和焦点到摄像机的距离;其次,通过坐标系变换,求解使用者注意焦点的世界坐标;再次,基于几何法求解机器人本体运动学逆解;最后,根据求得的关节角控制舵机,实现人机共同注意.通过仿真与物理平台的实验,验证了方法的有效性和准确性,为人机交互与合作中共同注意的实现提供了解决方案.     

Digital Twin for Human-Robot Interactive Welding and Welder Behavior Analysis

This paper presents an innovative investigation on prototyping a digital twin(DT)as the platform for human-robot interactive welding and welder behavior analysis.This humanrobot interaction(HRI)working style helps to enhance human users'operational productivity and comfort;while data-driven welder behavior analysis benefits to further novice welder training.This HRI system includes three modules:1)a human user who demonstrates the welding operations offsite with her/his operations recorded by the motion-tracked handles;2)a robot that executes the demonstrated welding operations to complete the physical welding tasks onsite;3)a DT system that is developed based on virtual reality(VR)as a digital replica of the physical human-robot interactive welding environment.The DT system bridges a human user and robot through a bi-directional information flow:a)transmitting demonstrated welding operations in VR to the robot in the physical environment;b)displaying the physical welding scenes to human users in VR.Compared to existing DT systems reported in the literatures,the developed one provides better capability in engaging human users in interacting with welding scenes,through an augmented VR.To verify the effectiveness,six welders,skilled with certain manual welding training and unskilled without any training,tested the system by completing the same welding job;three skilled welders produce satisfied welded workpieces,while the other three unskilled do not.A data-driven approach as a combination of fast Fourier transform(FFT),principal component analysis(PCA),and support vector machine(SVM)is developed to analyze their behaviors.Given an operation sequence,i.e.,motion speed sequence of the welding torch,frequency features are firstly extracted by FFT and then reduced in dimension through PCA,which are finally routed into SVM for classification.The trained model demonstrates a 94.44%classification accuracy in the testing dataset.The successful pattern recognition in skilled welder operations should benefit to accelerate novice welder training.     

Data-Driven Human-Robot Interaction Without Velocity Measurement Using Off-Policy Reinforcement Learning

In this paper, we present a novel data-driven design method for the human-robot interaction (HRI) system, where a given task is achieved by cooperation between the human and the robot. The presented HRI controller design is a two-level control design approach consisting of a task-oriented performance optimization design and a plant-oriented impedance controller design. The task-oriented design minimizes the human effort and guarantees the perfect task tracking in the outer-loop, while the plant-oriented achieves the desired impedance from the human to the robot manipulator end-effector in the inner-loop. Data-driven reinforcement learning techniques are used for performance optimization in the outer-loop to assign the optimal impedance parameters. In the inner-loop, a velocity-free filter is designed to avoid the requirement of end-effector velocity measurement. On this basis, an adaptive controller is designed to achieve the desired impedance of the robot manipulator in the task space. The simulation and experiment of a robot manipulator are conducted to verify the efficacy of the presented HRI design framework.      

Skill learning framework for human–robot interaction and manipulation tasks

In this article, a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed. The learning framework makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way. This approach offers human tutors the ability to control all joints of the robotic manipulator in real-time and able to achieve complex manipulation. The robotic manipulator is controlled remotely with our low-cost wearable devices for easy calibration and continuous motion mapping. We believe that our approach might lead to improving the human-robot skill learning, adaptability, and sensitivity of the proposed human-robot interaction for flexible task execution and thereby giving room for skill transfer and repeatability without complex coding skills.     

User Modeling for Adaptive News Access

We present a framework for adaptive news access, based on machine learning techniques specifically designed for this task. First, we focus on the system's general functionality and system architecture. We then describe the interface and design of two deployed news agents that are part of the described architecture. While the first agent provides personalized news through a web-based interface, the second system is geared towards wireless information devices such as PDAs (personal digital assistants) and cell phones. Based on implicit and explicit user feedback, our agents use a machine learning algorithm to induce individual user models. Motivated by general shortcomings of other user modeling systems for Information Retrieval applications, as well as the specific requirements of news classification, we propose the induction of hybrid user models that consist of separate models for short-term and long-term interests. Furthermore, we illustrate how the described algorithm can be used to address an important issue that has thus far received little attention in the Information Retrieval community: a user's information need changes as a direct result of interaction with information. We empirically evaluate the system's performance based on data collected from regular system users. The goal of the evaluation is not only to understand the performance contributions of the algorithm's individual components, but also to assess the overall utility of the proposed user modeling techniques from a user perspective. Our results provide empirical evidence for the utility of the hybrid user model, and suggest that effective personalization can be achieved without requiring any extra effort from the user.     

Model reference adaptive impedance control for physical human-robot interaction

This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.     

Teaching a pet-robot to understand user feedback through interactive virtual training tasks

In this paper, we present a human-robot teaching framework that uses “virtual” games as a means for adapting a robot to its user through natural interaction in a controlled environment. We present an experimental study in which participants instruct an AIBO pet robot while playing different games together on a computer generated playfield. By playing the games and receiving instruction and feedback from its user, the robot learns to understand the user’s typical way of giving multimodal positive and negative feedback. The games are designed in such a way that the robot can reliably predict positive or negative feedback based on the game state and explore its user’s reward behavior by making good or bad moves. We implemented a two-staged learning method combining Hidden Markov Models and a mathematical model of classical conditioning to learn how to discriminate between positive and negative feedback. The system combines multimodal speech and touch input for reliable recognition. After finishing the training, the system was able to recognize positive and negative reward with an average accuracy of 90.33%.     

自适应网络学习用户界面设计和实现

An adaptive user interface helps to improve the quality of human-computer interaction. Most people at present join to Web-Based Learning by common browser. Due to the one-fits-all user interface, they have to face with the problem of lack of the support on personalized learning. The design and implementation of the adaptive user interface for Web-based learning in this paper is grounded in our work done before, for example interaction model,adaptive user models including domain models. The adaptivity is mainly expressed on learning contents and representation including layout as well as operation.     

网络机器人的可调整自主性

Internet上数据传输的不确定延时妨碍了机器人和操作者之间迅捷而透明地交互, 严重限制了遥操作机器人的性能和应用. 本文研究了机器人的可调整自主性(Adjustable autonomy, AA), 通过改善人机交互来补偿网络通讯存在的不确定延时对系统性能的影响. 机器人的自主性根据当前形势和环境动态调整, 操作者和机器人以适合网络状况和任务需要的模式进行交互和合作, 使得整个系统的效率大大提高. 实验结果证明了所提方法的有效性和可行性.     

Pre-collision safety strategies for human-robot interaction

Safe planning and control is essential to bringing human-robot interaction into common experience. This paper presents an integrated human−robot interaction strategy that ensures the safety of the human participant through a coordinated suite of safety strategies that are selected and implemented to anticipate and respond to varying time horizons for potential hazards and varying expected levels of interaction with the user. The proposed planning and control strategies are based on explicit measures of danger during interaction. The level of danger is estimated based on factors influencing the impact force during a human-robot collision, such as the effective robot inertia, the relative velocity and the distance between the robot and the human. A second key requirement for improving safety is the ability of the robot to perceive its environment, and more specifically, human behavior and reaction to robot movements. This paper also proposes and demonstrates the use of human monitoring information based on vision and physiological sensors to further improve the safety of the human robot interaction. A methodology for integrating sensor-based information about the user's position and physiological reaction to the robot into medium and short-term safety strategies is presented. This methodology is verified through a series of experimental test cases where a human and an articulated robot respond to each other based on the human's physical and physiological behavior.

一种上下文移动用户偏好自适应学习方法

针对移动网络对个性化移动网络服务系统的性能提出了更高的要求,但现有研究难以自适应地修改上下文移动用户偏好以为移动用户提供实时、准确的个性化移动网络服务的问题,提出了一种上下文移动用户偏好自适应学习方法,在保证精确度的基础上缩短了学习的响应时间.首先,通过分析移动用户行为日志来判断移动用户行为是否受上下文影响,并在此基础上判断移动用户行为是否发生变化.然后,根据判断结果对上下文移动用户偏好进行修正.在对发生变化的上下文移动用户偏好进行学习时,将上下文引入到最小二乘支持向量机中,进一步提出了基于上下文最小二乘支持向量机(C-LSSVM)的上下文移动用户偏好学习方法.最后,实验结果表明,当综合考虑精确度和响应时间两方面因素时,所提出的方法优于其他学习方法,并且可应用于个性化移动网络服务系统中.     

Physics-based simulation for manual robot guidance—An eRobotics approach

Manual robot guidance is an intuitive approach to teach robots with human's skills in the loop. It is particularly useful to manufacturers because of its high flexibility and low programming effort. However, manual robot guidance requires compliance control that is generally not available in position-controlled industrial robots. We address this issue from a simulation-driven approach. We systematically capture the interactive dynamic behavior of intelligent robot manipulators within physics-based virtual testbeds, regardless of the type of application. On this basis, we develop structures to equip and employ simulated robots with motion control capabilities that include soft physical interaction control driven in real-time with real external guidance forces. We then transfer the virtual compliant behavior of the simulated robots to their physical counterparts to enable manual guidance. The simulator provides assistance to operators through timely and insightful robot monitoring, as well as meaningful performance indexes. The testbed allows us to swiftly assess guidance within numerous interaction scenarios. Experimental case studies illustrate the practical usefulness of the symbiotic transition between 3D simulation and reality, as pursued by the eRobotics framework to address challenging issues in industrial automation.     

Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot

In this paper, we present our work in building technologies for natural multimodal human-robot interaction. We present our systems for spontaneous speech recognition, multimodal dialogue processing, and visual perception of a user, which includes localization, tracking, and identification of the user, recognition of pointing gestures, as well as the recognition of a person's head orientation. Each of the components is described in the paper and experimental results are presented. We also present several experiments on multimodal human-robot interaction, such as interaction using speech and gestures, the automatic determination of the addressee during human-human-robot interaction, as well on interactive learning of dialogue strategies. The work and the components presented here constitute the core building blocks for audiovisual perception of humans and multimodal human-robot interaction used for the humanoid robot developed within the German research project (Sonderforschungsbereich) on humanoid cooperative robots.     

Skill-based human–robot cooperation in tele-operated path tracking

This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.     

基于语义网络的人体经络用户界面研究

将语义网络用于表征人体经络系统中复杂概念之间的关系,构建经络知识库,为建立智能化人机交互提供基础;同时将语义网络引入到系统用户界面设计,为信息用户提供智能引导,使系统能够根据用户的真实需求,为用户提供主动地、有针对性的个性化导航服务,提高用户的学习效率。     

Evolving behavior sequences for a humanoid entertainment robot

One of the most important issues in developing an entertainment robot is human-robot interaction, in which the robot is expected to learn new behaviors specified by the user. In this article we present an imitation-based mechanism to support robot learning, and use evolutionary computing to learn new behavior sequences. We also propose several advanced techniques at the task level and the computational level to evolve complex sequences. To evaluate our approach, we use it to evolve different behaviors for a humanoid robot. The results show the promise of our approach.     

基于情感-时空信息的机器人服务自主认知及个性化选择

为了提高机器人的人机交互能力,针对家庭服务机器人在认知服务任务时往往忽略用户情感因素的弊端,提出了以用户情感为核心的机器人服务任务自主认知方法以及个性化服务选择策略.首先,利用智能空间本体技术结合用户情感状态与时间空间信息建立情感-时空本体模型,消除智能空间中的信息异构性.在此基础上,将与情感-时空相关的服务规则库编码并训练BP（逆向传播）神经网络构建推理机,将实时更新的智能空间信息与神经网络相匹配,推理出机器人需要执行的服务,实现机器人对以用户情感为核心的服务任务自主认知.最后,将用户情感状态作为执行服务的奖惩反馈信号,对服务集合中的子类服务进行动态的偏好度调节,完成有针对性的服务选择.仿真结果表明,基于该方法能够实现以用户情感为核心的机器人服务任务自主认知,同时可以根据用户偏好变化提供个性化的服务,有效提高了家庭服务机器人的智能性和灵活性,增强了用户的服务体验.     

Full-Body Compliant Human–Humanoid Interaction: Balancing in the Presence of Unknown External Forces

This paper proposes an effective framework of human-humanoid robot physical interaction. Its key component is a new control technique for full-body balancing in the presence of external forces, which is presented and then validated empirically. We have adopted an integrated system approach to develop humanoid robots. Herein, we describe the importance of replicating human-like capabilities and responses during human-robot interaction in this context. Our balancing controller provides gravity compensation, making the robot passive and thereby facilitating safe physical interactions. The method operates by setting an appropriate ground reaction force and transforming these forces into full-body joint torques. It handles an arbitrary number of force interaction points on the robot. It does not require force measurement at interested contact points. It requires neither inverse kinematics nor inverse dynamics. It can adapt to uneven ground surfaces. It operates as a force control process, and can therefore, accommodate simultaneous control processes using force-, velocity-, or position-based control. Forces are distributed over supporting contact points in an optimal manner. Joint redundancy is resolved by damping injection in the context of passivity. We present various force interaction experiments using our full-sized bipedal humanoid platform, including compliant balance, even when affected by unknown external forces, which demonstrates the effectiveness of the method.     

人群环境中基于深度强化学习的移动机器人避障算法

为了控制移动机器人在人群密集的复杂环境中高效友好地完成避障任务,本文提出了一种人群环境中基于深度强化学习的移动机器人避障算法。首先,针对深度强化学习算法中值函数网络学习能力不足的情况,基于行人交互（crowd interaction）对值函数网络做了改进,通过行人角度网格（angel pedestrian grid）对行人之间的交互信息进行提取,并通过注意力机制（attention mechanism）提取单个行人的时序特征,学习得到当前状态与历史轨迹状态的相对重要性以及对机器人避障策略的联合影响,为之后多层感知机的学习提供先验知识;其次,依据行人空间行为（human spatial behavior）设计强化学习的奖励函数,并对机器人角度变化过大的状态进行惩罚,实现了舒适避障的要求;最后,通过仿真实验验证了人群环境中基于深度强化学习的移动机器人避障算法在人群密集的复杂环境中的可行性与有效性。     

User demographics prediction based on mobile data

Demographics prediction is an important component of user profile modeling. The accurate prediction of users’ demographics can help promote many applications, ranging from web search, personalization to behavior targeting. In this paper, we focus on how to predict users’ demographics, including “gender”, “job type”, “marital status”, “age” and “number of family members”, based on mobile data, such as users’ usage logs, physical activities and environmental contexts. The core idea is to build a supervised learning framework, where each user is represented as a feature vector and users’ demographics are considered as prediction targets. The most important component is to construct features from raw data and then supervised learning models can be applied. We propose a feature construction framework, CFC (contextual feature construction), where each feature is defined as the conditional probability of one user activity under the given contexts. Consequently, besides employing standard supervised learning models, we propose a regularized multi-task learning framework to model different kinds of demographics predictions collectively. We also propose a cost-sensitive classification framework for regression tasks, in order to benefit from the existing dimension reduction methods. Finally, due to the limited training instances, we employ ensemble to avoid overfitting. The experimental results show that the framework achieves classification accuracies on “gender”, “job” and “marital status” as high as 96%, 83% and 86%, respectively, and achieves Root Mean Square Error (RMSE) on “age” and “number of family members” as low as 0.69 and 0.66 respectively, under the leave-one-out evaluation.