A two-dimensional facial-affect estimation system for human–robot interaction using facial expression parameters

A social robot should be able to autonomously interpret human affect and adapt its behavior accordingly in order for successful social human–robot interaction to take place. This paper presents a modular non-contact automated affect-estimation system that employs support vector regression over a set of novel facial expression parameters to estimate a person’s affective states using a valence-arousal two-dimensional model of affect. The proposed system captures complex and ambiguous emotions that are prevalent in real-world scenarios by utilizing a continuous two-dimensional model, rather than a traditional discrete categorical model for affect. As the goal is to incorporate this recognition system in robots, real-time estimation of spontaneous natural facial expressions in response to environmental and interactive stimuli is an objective. The proposed system can be combined with affect detection techniques using other modes, such as speech, body language and/or physiological signals, etc., in order to develop an accurate multi-modal affect estimation system for social HRI applications. Experiments presented herein demonstrate the system’s ability to successfully estimate the affect of a diverse group of unknown individuals exhibiting spontaneous natural facial expressions.     

Posture control of robot manipulators with fuzzy voice commands using a fuzzy coach–player system

This paper presents a method of controlling robot manipulators with fuzzy voice commands. Recently, there has been some research on controlling robots using information-rich fuzzy voice commands such as 'go little slowly' and learning from such commands. However, the scope of all those works was limited to basic fuzzy voice motion commands. In this paper, we introduce a method of controlling the posture of a manipulator using complex fuzzy voice commands. A complex fuzzy voice command is composed of a set of fuzzy voice joint commands. Complex fuzzy voice commands can be used for complicated maneuvering of a manipulator, while fuzzy voice joint commands affect only a single joint. Once joint commands are learned, any complex command can be learned as a combination of some or all of them, so that, using the learned complex commands, a human user can control the manipulator in a complicated manner with natural language commands. Learning of complex commands is discussed in the framework of fuzzy coach–player model. The proposed idea is demonstrated with a PA-10 redundant manipulator.     

A biologically constrained architecture for developmental learning of eye–head gaze control on a humanoid robot

In this paper we describe a biologically constrained architecture for developmental learning of eye–head gaze control on an iCub robot. In contrast to other computational implementations, the developmental approach aims to acquire sensorimotor competence through growth processes modelled on data and theory from infant psychology. Constraints help shape learning in infancy by limiting the complexity of interactions between the body and environment, and we use this idea to produce efficient, effective learning in autonomous robots. Our architecture is based on current thinking surrounding the gaze mechanism, and experimentally derived models of stereotypical eye–head gaze contributions. It is built using our proven constraint-based field-mapping approach. We identify stages in the development of infant gaze control, and propose a framework of artificial constraints to shape learning on the robot in a similar manner. We demonstrate the impact these constraints have on learning, and the resulting ability of the robot to make controlled gaze shifts.     

A secure and efficient identity-based authenticated key exchange protocol for mobile client–server networks

Recently, Chou et al. (J Supercomput 66(2): 973–988, 2013) proposed two identity-based key exchange protocols using elliptic curves for mobile environments. The first one is an two-party authentication key exchange protocol to establish a session key between a client and a remote server. The second one is an extended version for three-party setting to establish a session key between two clients with the help of a trusted server. However, this paper finds the first one vulnerable to impersonation attack and key-compromise impersonation attack, and the second one insecure against impersonation attack. To overcome the weaknesses, we propose an improved identity-based two-party authentication key exchange protocol using elliptic curves. The rigorous analysis shows that our scheme achieves more security than related protocols.     

Shared control architecture based on RFID to control a robot arm using a spontaneous brain–machine interface

This paper describes a shared control architecture combining a Brain–Machine Interface (BMI) with Radio-frequency Identification (RFID) technology to control a robot arm in pick and place operations. A non-invasive spontaneous BMI capable of distinguishing between three different mental tasks has been designed. Using the BMI, the user can control the robot in order to perform complex actions (e.g. pick and place operations). RFID tags have been placed in the experimental setup to give information about the position of the objects in the scene. With this information, the user is able to pick and place the objects with a robot arm by performing simple commands: move left, move right, pick or place, with the only help of the BMI. Four volunteers have successfully controlled the robot arm, and time and accuracy have been measured.     

Cryptanalysis of a remote user authentication scheme for mobile client–server environment based on ECC

Understanding security failures of cryptographic protocols is the key to both patching existing protocols and designing future schemes. The design of secure remote user authentication schemes based on elliptic curve crypto-graphy (ECC) for mobile applications is still quite a challenging problem, though many schemes have been published lately. In this paper, we analyze an efficient ID-based scheme for mobile client–server environment without the MapToPoint function introduced by He et al. in 2012. This proposal attempts to overcome many of the well known security and efficiency shortcomings of previous schemes, and it also carries a claimed proof of security in the random oracle model. However, notwithstanding its formal security arguments, we show that He et al.’s protocol even cannot attain the basic goal of mutual authentication by demonstrating its vulnerabilities to reflection attack and parallel session attack. Besides these two security vulnerabilities, their scheme also suffers from some practical pitfalls such as user anonymity violation and clock synchronization problem. In addition, we carry out an investigation into their security proof and propose some changes to the scheme so that it can achieve at least its basic security goal, in the hope that similar mistakes are no longer made in the future.     

Markerless human–robot interface for dual robot manipulators using Kinect sensor

Remote teleoperation of robot manipulators is often necessary in unstructured, dynamic, and dangerous environments. However, the existing mechanical and other contacting interfaces require unnatural, or hinder natural, human motions. At present, the contacting interfaces used in teleoperation for multiple robot manipulators often require multiple operators. Previous vision-based approaches have only been used in the remote teleoperation for one robot manipulator as well as require the special quantity of illumination and visual angle that limit the field of application. This paper presents a noncontacting Kinect-based method that allows a human operator to communicate his motions to the dual robot manipulators by performing double hand–arm movements that would naturally carry out an object manipulation task. This paper also proposes an innovative algorithm of over damping to solve the problem of error extracting and dithering due to the noncontact measure. By making full use of the human hand–arm motion, the operator would feel immersive. This human–robot interface allows the flexible implementation of the object manipulation task done in collaboration by dual robots through the double hand–arm motion by one operator.     

Human–robot skills transfer interfaces for a flexible surgical robot

In minimally invasive surgery, tools go through narrow openings and manipulate soft organs to perform surgical tasks. There are limitations in current robot-assisted surgical systems due to the rigidity of robot tools. The aim of the STIFF-FLOP European project is to develop a soft robotic arm to perform surgical tasks. The flexibility of the robot allows the surgeon to move within organs to reach remote areas inside the body and perform challenging procedures in laparoscopy. This article addresses the problem of designing learning interfaces enabling the transfer of skills from human demonstration. Robot programming by demonstration encompasses a wide range of learning strategies, from simple mimicking of the demonstrator's actions to the higher level imitation of the underlying intent extracted from the demonstrations. By focusing on this last form, we study the problem of extracting an objective function explaining the demonstrations from an over-specified set of candidate reward functions, and using this information for self-refinement of the skill. In contrast to inverse reinforcement learning strategies that attempt to explain the observations with reward functions defined for the entire task (or a set of pre-defined reward profiles active for different parts of the task), the proposed approach is based on context-dependent reward-weighted learning, where the robot can learn the relevance of candidate objective functions with respect to the current phase of the task or encountered situation. The robot then exploits this information for skills refinement in the policy parameters space. The proposed approach is tested in simulation with a cutting task performed by the STIFF-FLOP flexible robot, using kinesthetic demonstrations from a Barrett WAM manipulator.     

Approximate analysis of a queueing–inventory system with early and delayed server vacations

We propose a model for a servicing system with perishable inventory and a finite queue of impatient claims where the server can be in one of three states: operational, early and delaying vacations. We develop a method for approximate computation of the system’s characteristics. We show results of numerical experiments.     

Remote human–robot collaboration: A cyber–physical system application for hazard manufacturing environment

Collaborative robot's lead-through is a key feature towards human–robot collaborative manufacturing. The lead-through feature can release human operators from debugging complex robot control codes. In a hazard manufacturing environment, human operators are not allowed to enter, but the lead-through feature is still desired in many circumstances. To target the problem, the authors introduce a remote human–robot collaboration system that follows the concept of cyber–physical systems. The introduced system can flexibly work in four different modes according to different scenarios. With the utilisation of a collaborative robot and an industrial robot, a remote robot control system and a model-driven display system is designed. The designed system is also implemented and tested in different scenarios. The final analysis indicates a great potential to adopt the developed system in hazard manufacturing environment.     

A human–robot interface for mobile manipulator

This paper presents a remote manipulation method for mobile manipulator through operator’s gesture. In particular, a track mobile robot is equipped with a 4-DOF robot arm to grasp objects. Operator uses one hand to control both the motion of mobile robot and the posture of robot arm via scheme of gesture polysemy method which is put forward in this paper. A sensor called leap motion (LM), which can obtain the position and posture data of hand, is employed in this system. Two filters were employed to estimate the position and posture of human hand so as to reduce the inherent noise of the sensor. Kalman filter was used to estimate the position, and particle filter was used to estimate the orientation. The advantage of the proposed method is that it is feasible to control a mobile manipulator through just one hand using a LM sensor. The effectiveness of the proposed human–robot interface was verified in laboratory with a series of experiments. And the results indicate that the proposed human–robot interface is able to track the movements of operator’s hand with high accuracy. It is found that the system can be employed by a non-professional operator for robot teleoperation.     

An efficient user authentication and key exchange protocol for mobile client–server environment

Considering the low-power computing capability of mobile devices, the security scheme design is a nontrivial challenge. The identity (ID)-based public-key system with bilinear pairings defined on elliptic curves offers a flexible approach to achieve simplifying the certificate management. In the past, many user authentication schemes with bilinear pairings have been proposed. In 2009, Goriparthi et al. also proposed a new user authentication scheme for mobile client–server environment. However, these schemes do not provide mutual authentication and key exchange between the client and the server that are necessary for mobile wireless networks. In this paper, we present a new user authentication and key exchange protocol using bilinear pairings for mobile client–server environment. As compared with the recently proposed pairing-based user authentication schemes, our protocol provides both mutual authentication and key exchange. Performance analysis is made to show that our presented protocol is well suited for mobile client–server environment. Security analysis is given to demonstrate that our proposed protocol is provably secure against previous attacks.     

A proxy-based integrated cache consistency and mobility management scheme for client–server applications in Mobile IP systems

In this paper, we investigate a proxy-based integrated cache consistency and mobility management scheme for supporting client–server applications in Mobile IP systems with the objective to minimize the overall network traffic generated. Our cache consistency management scheme is based on a stateful strategy by which cache invalidation messages are asynchronously sent by the server to a mobile host (MH) whenever data objects cached at the MH have been updated. We use a per-user proxy to buffer invalidation messages to allow the MH to disconnect arbitrarily and to reduce the number of uplink requests when the MH is reconnected. Moreover, the user proxy takes the responsibility of mobility management to further reduce the network traffic. We investigate a design by which the MH’s proxy serves as a gateway foreign agent (GFA) as in the MIP Regional Registration protocol to keep track of the address of the MH in a region, with the proxy migrating with the MH when the MH crosses a regional area. We identify the optimal regional area size under which the overall network traffic cost, due to cache consistency management, mobility management, and query requests/replies, is minimized. The integrated cache consistency and mobility management scheme is demonstrated to outperform MIPv6, no-proxy and/or no-cache schemes, as well as a decoupled scheme that optimally but separately manages mobility and service activities in Mobile IPv6 environments.     

An explicit form for the closed-loop system matrix of a partially uncontrollable linear system†

An explicit expression for the closed-loop system matrix of a partially uncontrollable linear system is presented. This matrix shows very clearly the eigenproperties of the closed-loop system and, in particular, the relationship between controllability and eigenvalue assignment.     

Control of a 2 DoF robot using a Brain–Machine Interface

In this paper, a non-invasive spontaneous Brain–Machine Interface (BMI) is used to control the movement of a planar robot. To that end, two mental tasks are used to manage the visual interface that controls the robot. The robot used is a PupArm, a force-controlled planar robot designed by the nBio research group at the Miguel Hernández University of Elche (Spain). Two control strategies are compared: hierarchical and directional control. The experimental test (performed by four users) consists of reaching four targets. The errors and time used during the performance of the tests are compared in both control strategies (hierarchical and directional control). The advantages and disadvantages of each method are shown after the analysis of the results. The hierarchical control allows an accurate approaching to the goals but it is slower than using the directional control which, on the contrary, is less precise. The results show both strategies are useful to control this planar robot. In the future, by adding an extra device like a gripper, this BMI could be used in assistive applications such as grasping daily objects in a realistic environment. In order to compare the behavior of the system taking into account the opinion of the users, a NASA Tasks Load Index (TLX) questionnaire is filled out after two sessions are completed.     

A rescue robot system for collecting information designed for ease of use — a proposal of a rescue systems concept

A remote controlled robot for collecting information in disasters, e.g. earthquakes, is one of most effective applications of robots, because it is very dangerous for human beings to locate survivors in collapsed buildings and, in addition, small robots can move into narrow spaces to find survivors. However, previous rescue systems that use robots have a significant problem — a shortage of operators. In catastrophic disasters, in order to save victims, we must explore wide areas within a limited time. Thus, many rescue robots should be employed simultaneously. However, human interfaces of previous rescue robots were complicated, so that well-trained professional operators were needed to operate the robots and, thus, to use many rescue robots, many professional operators were required. However, in such catastrophic disasters it is difficult to get many professional operators together within a short time. In this paper we address the problem and propose a concept of rescue team organization in which professional rescue staff and volunteer staff work together for handling a catastrophic disaster. We point out the necessity for rescue robots which can be operated easily by non-professional volunteer staff. To realize a rescue robot which can be operated easily, we propose a rescue robot system which has a human interface seen in typical, everyday vehicles and a snake-like robot which has mechanical intelligence. We have demonstrated the validity and the effectiveness of the proposed concept by developing a prototype system.     

Adaptation of robot’s perception of fuzzy linguistic information by evaluating vocal cues for controlling a robot manipulator

This article proposes a method for adapting a robot’s perception of fuzzy linguistic information by evaluating vocal cues. The robot’s perception of fuzzy linguistic information such as “very little” depends on the environmental arrangements and the user’s expectations. Therefore, the robot’s perception of the corresponding environment is modified by acquiring the user’s perception through vocal cues. Fuzzy linguistic information related to primitive movements is evaluated by a behavior evaluation network (BEN). A vocal cue evaluation system (VCES) is used to evaluate the vocal cues for modifying the BEN. The user’s satisfactory level for the robot’s movements and the user’s willingness to change the robot’s perception are identified based on a series of vocal cues to improve the adaptation process. A situation of cooperative rearrangement of the user’s working space is used to illustrate the proposed system by a PA-10 robot manipulator.     

Remote user authentication and key agreement for mobile client–server environments on elliptic curve cryptography

In recent years, with the rapid advance of wireless mobile networks, secure and efficient authentication mechanisms that can operate over insecure wireless channels have become increasingly essential. To improve the efficiency in the energy-limited mobile devices, many authentication schemes using elliptic curve cryptography (ECC) have been presented. However, these schemes are still inefficient in terms of computation cost and communication overhead. Moreover, they suffer from various attacks, making them impractical due to their inherent design. To address their weaknesses, we propose a more efficient ID-based authentication scheme on ECC for mobile client–server environments with considering security requirements. The proposed scheme not only provides mutual authentication but also achieves session key agreement between the client and the server. Through a rigorous formal security proof under random oracle model, it has been indicated that the proposed protocol is secure against security threats. The informal security analysis shows that our scheme can resist well-known attacks and provides user anonymity. Performance analysis and comparison results demonstrate that our scheme outperforms the related competitive works and is more suitable for practical application in mobile client–server environments.     

A unified multimodal control framework for human–robot interaction

In human–robot interaction, the robot controller must reactively adapt to sudden changes in the environment (due to unpredictable human behaviour). This often requires operating different modes, and managing sudden signal changes from heterogeneous sensor data. In this paper, we present a multimodal sensor-based controller, enabling a robot to adapt to changes in the sensor signals (here, changes in the human collaborator behaviour). Our controller is based on a unified task formalism, and in contrast with classical hybrid visicn–force–position control, it enables smooth transitions and weighted combinations of the sensor tasks. The approach is validated in a mock-up industrial scenario, where pose, vision (from both traditional camera and Kinect), and force tasks must be realized either exclusively or simultaneously, for human–robot collaboration.