Incremental learning of gestures for human–robot interaction

For a robot to cohabit with people, it should be able to learn people’s nonverbal social behavior from experience. In this paper, we propose a novel machine learning method for recognizing gestures used in interaction and communication. Our method enables robots to learn gestures incrementally during human–robot interaction in an unsupervised manner. It allows the user to leave the number and types of gestures undefined prior to the learning. The proposed method (HB-SOINN) is based on a self-organizing incremental neural network and the hidden Markov model. We have added an interactive learning mechanism to HB-SOINN to prevent a single cluster from running into a failure as a result of polysemy of being assigned more than one meaning. For example, a sentence: “Keep on going left slowly” has three meanings such as, “Keep on (1)”, “going left (2)”, “slowly (3)”. We experimentally tested the clustering performance of the proposed method against data obtained from measuring gestures using a motion capture device. The results show that the classification performance of HB-SOINN exceeds that of conventional clustering approaches. In addition, we have found that the interactive learning function improves the learning performance of HB-SOINN.     

Skill learning framework for human–robot interaction and manipulation tasks

In this article, a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed. The learning framework makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way. This approach offers human tutors the ability to control all joints of the robotic manipulator in real-time and able to achieve complex manipulation. The robotic manipulator is controlled remotely with our low-cost wearable devices for easy calibration and continuous motion mapping. We believe that our approach might lead to improving the human-robot skill learning, adaptability, and sensitivity of the proposed human-robot interaction for flexible task execution and thereby giving room for skill transfer and repeatability without complex coding skills.     

One-shot learning of human–robot handovers with triadic interaction meshes

We propose an imitation learning methodology that allows robots to seamlessly retrieve and pass objects to and from human users. Instead of hand-coding interaction parameters, we extract relevant information such as joint correlations and spatial relationships from a single task demonstration of two humans. At the center of our approach is an interaction model that enables a robot to generalize an observed demonstration spatially and temporally to new situations. To this end, we propose a data-driven method for generating interaction meshes that link both interaction partners to the manipulated object. The feasibility of the approach is evaluated in a within user study which shows that human–human task demonstration can lead to more natural and intuitive interactions with the robot.     

Detecting human—object interaction with multi-level pairwise feature network

Computational Visual Media - Human–object interaction (HOI) detection is crucial for human-centric image understanding which aims to infer ⟨human, action, object⟩ triplets...     

Simulation-based learning: The learning–forgetting–relearning process and impact of learning history

The results of empirical experiments evaluating the effectiveness and efficiency of the learning–forgetting–relearning process in a dynamic project management simulation environment are reported. Sixty-six graduate engineering students performed repetitive simulation-runs with a break period of several weeks between the runs. The students used a teaching tool called the project management trainer (PMT) that simulates a generic dynamic, stochastic project management environment. In this research, we focused on the effect of history recording mechanism on the learning forgetting process. Manual or automatic history recording mechanisms were used by the experimental group, while the control group did not use any history recording mechanism. The findings indicate that for the initial learning phase, the manual mechanism is better than the automatic mechanism. However, for the relearning phase, the break period length influenced the performance after the break. When the break period is short, the manual history keeping mechanism is better, but for a long period break, there is no significant difference. A comparison between the experimental group and the control group revealed that using any history recording mechanism reduced forgetting. Based on the findings, some practical implications of using simulators to improve the learning–forgetting process are discussed.     

Where do machine learning and human-computer interaction meet?

Implementation of machine learning (ML) in human-computer interaction (HCI) work is not trivial. This article reports on a survey of 112 professionals and academicians specializing in HCI, who were asked to state level of ML use in HCI work. Responses were captured via a structured questionnaire. Analysis showed that about one-third of those who participated in the survey had used ML in conjunction with a variety of different HCI tasks. However, statistically significant differences could not be identified between those who have and those who have not used ML. Using statistics, contingency analysis, and clustering, we modeled interaction between representative HCI tasks and ML paradigms. We discovered that neural networks, rule induction, and statistical learning emerged as the most popular ML paradigms across HCI workers, although intensive learning, such as inductive logic programming, are gaining popularity among application developers. We also discovered that the leading causes for declining use of ML in HCI work are (1) misperceptions about ML, (2) lack of awareness of ML's potential, and (3) scarcity of concrete case studies demonstrating the application of ML in HCI.     

Virtual reality human–robot interaction technology acceptance model for learning direct current and alternating current

In this study, we have developed a set of virtual reality (VR) human–robot interaction technology acceptance model for learning direct current and alternating current, aiming to use VR technology to immerse students in the generation, existence, and flow of electricity. We hope that using VR to transform abstract physical concepts into tangible objects will help students learn and comprehend abstract electrical concepts. The VR technology acceptance model was developed using the Unity 3D game kit to be accessed using the HTC Vive VR headset. The scene models, characters, and objects were created using Autodesk 3DS Max and Autodesk Maya, and the 2D graphics were processed in Adobe Photoshop. The results were evaluated using four metrics for our technology acceptance model. The four metrics include the content, design, interface and media content, and practical requirements. The average score of the content is 4.73. The average score of the design is 4.12. The average score of the interface and media content is 4.34. The average score of the practical requirements is 3.72. All the items on the effectiveness questionnaire of the technology acceptance model had average scores in the range 4.25–4.75. Therefore, all teachers were strongly satisfied with the trial teaching activity. The average score of each statement ranged within 3.58–4.03 for the satisfaction with the teaching material contents. Hence, the students were somewhat satisfied with this teaching activity. The average score of each statement ranged from 3.43 to 4.96 for the satisfaction with the implementation of the technology acceptance model. This result shows that the respondents were generally satisfied with the learning outcomes associated with these materials. The average score per question in this questionnaire was 3.92, and most of the questions have an average score greater than 3.8 for the feedback pertaining to satisfaction with the teaching material contents. In summary, a deeply immersive and interactive game was created using tactile somatosensory devices and VR that aim to utilize and enhance the fun and benefits associated with learning from games.

     

Bridging the requirements–implementation modeling gap with object–process methodology

A model-based system development cycle involves two semantically distinct aspects: the requirements specification and the implementation model. Due to the conceptual and semantic differences between these two major system lifecycle stages, the transition from requirements to implementation is inherently a noncoherent process. Consequently, the system requirements are not faithfully transformed into the working system. This paper introduces an effective solution via an Integrated Modeling Paradigm (IMP) that combines the requirements and implementation domain models into a unified system model that continuously represents the system as it evolves. The IMP was implemented in an Object–Process Methodology (OPM) development environment. This implementation reinforces OPM with the capability to bridge the significant conceptual gap that lies right at the heart of the development process. A user survey has shown that this OPM-based solution is easy to use and can indeed help bridge the information gap, yielding a better match between the required and implemented systems than the currently accepted practice.     

Inkjet-printed proximity sensor for human–robot interaction

Microsystem Technologies - With the rapid development of artificial intelligence (AI), the human–robot interaction (HRI) has been attracted considerable attention in recent years. The...     

Driver assist system for human–machine interaction

A haptic driver–vehicle steering interface is introduced that interacts with the driver through environmentally mediated torque and stiffness changes, thereby communicating the vehicle’s proximity to constraints in the driving environment. The system design is based on principles of distributed cognition, which are used to shape the force characteristics to provide the driver with a tangible, rich, distributed representation of the task constraints. This mapping of the task constraints aids the human operator in satisfying a variety of needs, including managing risk, maintaining contextual awareness and achieving a satisfactory level of performance. The proposed design philosophy was applied to implement a haptic steering system in a real-world test vehicle to assist drivers in navigating through an experimental course with tight passages. In tests conducted with 12 participants, not only did most drivers show improved performance, but activities identified as epistemic behaviors were also observed; in the context of driving, epistemic behaviors are actions through which a driver probes the environment to maintain contextual awareness for the purpose of actively maintaining a satisfactory balance between performance and risk. These findings indicate that the proposed system design allows humans to actively integrate the haptic interface system into their cognitive loops and that the resulting human–machine system achieves higher performance than the human alone. The observed human–machine system interaction is interpreted as achieving improved resilience against variations in environmentally imposed risks.     

A human–computer interaction approach for healthcare

Universal Access in the Information Society -     

Efficient behavior learning in human–robot collaboration

We present a novel method for a robot to interactively learn, while executing, a joint human–robot task. We consider collaborative tasks realized by a team of a human operator and a robot helper that adapts to the human’s task execution preferences. Different human operators can have different abilities, experiences, and personal preferences so that a particular allocation of activities in the team is preferred over another. Our main goal is to have the robot learn the task and the preferences of the user to provide a more efficient and acceptable joint task execution. We cast concurrent multi-agent collaboration as a semi-Markov decision process and show how to model the team behavior and learn the expected robot behavior. We further propose an interactive learning framework and we evaluate it both in simulation and on a real robotic setup to show the system can effectively learn and adapt to human expectations.     

Four different perspectives on human–computer interaction

This paper will stress the value of a multi-perspective view on the use of computers. It will argue that the ability to apply more than one perspective is valuable to designers of computer applications, to researchers dealing with human-computer interactions, as well as to users of a particular computer application. As a means for that the paper will present the systems perspective, the dialogue partner perspective, the tool perspective, and the media perspective. All four perspectives will primarily be characterized in relation to human-computer interaction, and the characterizations will be based on a common set of concepts presented in the beginning of the paper. The last section of the paper will, with the help of a few examples, illustrate the value of applying multiple perspectives.     

A philosophical study of human–artefact interaction

This paper focuses on a critical intersection between philosophy of technology and cognitive archaeology with an objective in view. These two rapidly developing disciplines intersect on the problem of characterizing the dynamic relationship between human beings and technical artefacts. The intricacies of human–artefact relation have been a source of curiosity and contemplation for philosophers of technology since 1970s. In contrast, the cognitive archaeologists’ interest in interpreting the exact nature of the interaction between human cognitive system and material culture is relatively recent. The central objective of this paper is to show why the cognitive archaeologists’ account of the relation between human cognition and material culture as exemplified by the classical-phenomenological example of the blind person’s use of a stick needs to be critically reviewed in the light of philosophical-(post)-phenomenological research and new empirical findings on tool use and prosthesis. There are three sections in this paper. In the first section, certain distinctive features of cognitive archaeology, which are important for the following discussion, are mentioned in brief. The second section consists of an exposition of Don Ihde’s account of embodiment relations—typical examples of which include the blind person’s use of a stick—with particular emphasis on the aspect of what Ihde calls “quasi-transparency”. Possible reasons behind the cognitive archaeologists’ indifference to the said aspect are pointed out. In the third section, the difficulties involved in analysing the case of the blind person’s stick are discussed in the light of recent empirical research on bodily extension (by means of artefacts) and prosthesis (incorporation of artefacts into the body). The paper ends with some critical comments on the cognitive archaeologists’ interpretation of the relation between the blind person and his stick and explains why their interpretation requires revision in view of current findings on tool use and prosthesis.