System for augmented human–robot interaction through mixed reality and robot training by non-experts in customer service environments

Human–robot interaction during general service tasks in home or retail environment has been proven challenging, partly because (1) robots lack high-level context-based cognition and (2) humans cannot intuit the perception state of robots as they can for other humans. To solve these two problems, we present a complete robot system that has been given the highest evaluation score at the Customer Interaction Task of the Future Convenience Store Challenge at the World Robot Summit 2018, which implements several key technologies: (1) a hierarchical spatial concepts formation for general robot task planning and (2) a mixed reality interface to enable users to intuitively visualize the current state of the robot perception and naturally interact with it. The results obtained during the competition indicate that the proposed system allows both non-expert operators and end users to achieve human–robot interactions in customer service environments. Furthermore, we describe a detailed scenario including employee operation and customer interaction which serves as a set of requirements for service robots and a road map for development. The system integration and task scenario described in this paper should be helpful for groups facing customer interaction challenges and looking for a successfully deployed base to build on.     

A method of vertical and horizontal force estimation by using air-filled material and camera for soft physical human-robot interaction: fundamental experiments

A method of vertical and horizontal force estimation for soft physical human–robot interaction by using an air-filled material and a camera was proposed in this study. First, we considered hypotheses for vertical and horizontal force applied to the air-filled material. Second, the prototype of air-filled material and vision-based force-sensing device (AVFSD) was constructed. Based on the result of preliminary experiments, the relationships between force, pneumatic pressure, and contact surface displacement were formulated. The vertical and horizontal force could be estimated using the AVFSD with percentage errors of 3.6 and 5.8%, respectively, and the hypotheses were verified through the experiments.