Visual recognition of pointing gestures for human–robot interaction

In this paper, we present an approach for recognizing pointing gestures in the context of human–robot interaction. In order to obtain input features for gesture recognition, we perform visual tracking of head, hands and head orientation. Given the images provided by a calibrated stereo camera, color and disparity information are integrated into a multi-hypothesis tracking framework in order to find the 3D-positions of the respective body parts. Based on the hands’ motion, an HMM-based classifier is trained to detect pointing gestures. We show experimentally that the gesture recognition performance can be improved significantly by using information about head orientation as an additional feature. Our system aims at applications in the field of human–robot interaction, where it is important to do run-on recognition in real-time, to allow for robot egomotion and not to rely on manual initialization.     

An augmented reality human–computer interface for object localization in a cognitive vision system

The European Cognitive Vision project VAMPIRE uses mobile AR-kits to interact with a visual active memory for teaching and retrieval purposes. This paper describes concept and technical realization of the used mobile AR-kits and discusses interactive learning and retrieval in office environments, and the active memory infrastructure. The focus is on 3D interaction for pointing in a scene coordinate system. This is achieved by 3D augmented pointing, which combines inside-out tracking for head pose recovery and 3D stereo human–computer interaction. Experimental evaluation shows that the accuracy of this 3D cursor is within a few centimeters, which is sufficient to point at an object in an office. Finally, an application of the cursor in VAMPIRE is presented, where in addition to the mobile system, at least one stationary active camera is used to obtain different views of an object. There are many potential applications, for example an improved view-based object recognition.     

Shifting viewpoints: Artificial intelligence and human–computer interaction

The AI and HCI communities have often been characterized as having opposing views of how humans and computers should interact. As both of them evolve, there is a deeper contrast that cuts across these communities, in how researchers conceive the relationship between knowledge and design. By examining the rationalistic and design orientations underlying bodies of work in both disciplines, we highlight relevant differences and possibilities for effective interaction with computers.     

Integrating a flexible anthropomorphic, robot hand into the control, system of a humanoid robot

This article presents the approaches taken to integrate a novel anthropomorphic robot hand into a humanoid robot. The requisites enabling such a robot hand to use everyday objects in an environment built for humans are presented. Starting from a design that resembles the human hand regarding size and movability of the mechatronical system, a low-level control system is shown providing reliable and stable controllers for single joint angles and torques, entire fingers and several coordinated fingers. Further on, the high-level control system connecting the low-level control system with the rest of the humanoid robot is presented. It provides grasp skills to the superior robot control system, coordinates movements of hand and arm and determines grasp patterns, depending on the object to grasp and the task to execute. Finally some preliminary results of the system, which is currently tested in simulations, will be presented.     

A freehand-sketch environment for architectural design supported by a multi-agent system

This paper presents an on-line system for capturing and interpreting architectural sketches. The prototype is based on a multi-agent system, which enables real-time management of recognition scenarios. We describe the different types of agents, their characteristics, the basic mechanisms involved in interpreting freehand architectural drawings and the collaboration modes between agents. Finally, we illustrate the general operations of the system by a short example.     

The Personal Rover Project:: The comprehensive design of a domestic personal robot

In this paper, we summarize an approach for the dissemination of robotics technologies. In a manner analogous to the personal computer movement of the early 1980s, we propose that a productive niche for robotic technologies is as a long-term creative outlet for human expression and discovery. To this end, this paper describes our ongoing efforts to design, prototype and test a low-cost, highly competent personal rover for the domestic environment.     

Social and collaborative aspects of interaction with a service robot

To an increasing extent, robots are being designed to become a part of the lives of ordinary people. This calls for new models of the interaction between humans and robots, taking advantage of human social and communicative skills. Furthermore, human–robot relationships must be understood in the context of use of robots, and based on empirical studies of humans and robots in real settings. This paper discusses social aspects of interaction with a service robot, departing from our experiences of designing a fetch-and-carry robot for motion-impaired users in an office environment. We present the motivations behind the design of the Cero robot, especially its communication paradigm. Finally, we discuss experiences from a recent usage study, and research issues emerging from this work. A conclusion is that addressing only the primary user in service robotics is unsatisfactory, and that the focus should be on the setting, activities and social interactions of the group of people where the robot is to be used.     

Toward sociable robots

This paper explores the topic of social robots—the class of robots that people anthropomorphize in order to interact with them. From the diverse and growing number of applications for such robots, a few distinct modes of interaction are beginning to emerge. We distinguish four such classes: socially evocative, social interface, socially receptive, and sociable. For the remainder of the paper, we explore a few key features of sociable robots that distinguish them from the others. We use the vocal turn-taking behavior of our robot, Kismet, as a case study to highlight these points.     

A sensor for dynamic tactile information with applications in human–robot interaction and object exploration

We present a novel tactile sensor, which is applied for dextrous grasping with a simple robot gripper. The hardware novelty consists of an array of capacitive sensors, which couple to the object by means of little brushes of fibers. These sensor elements are very sensitive (with a threshold of about 5 mN) but robust enough not to be damaged during grasping. They yield two types of dynamical tactile information corresponding roughly to two types of tactile sensor in the human skin. The complete sensor consists of a foil-based static force sensor, which yields the total force and the center of the two-dimensional force distribution and is surrounded by an array of the dynamical sensor elements. One such sensor has been mounted on each of the two gripper jaws of our humanoid robot and equipped with the necessary read-out electronics and a CAN bus interface. We describe applications to guiding a robot arm on a desired trajectory with negligible force, reflective grip improvement, and tactile exploration of objects to create a shape representation and find stable grips, which are applied autonomously on the basis of visual recognition.     

An ethological and emotional basis for human–robot interaction

This paper presents the role of ethological and emotional models as the basis for an architecture in support of entertainment robotic systems. Specific examples for Sony’s AIBO are presented as well as extensions related to a new humanoid robot, SDR.