Development of an Intelligent Pneumatic Cylinder for Distributed Physical Human–Machine Interaction

Pneumatic systems are well known for their advantages and simplicity, and have been applied in various applications. This paper presents the development and experimental evaluation of an intelligent pneumatic cylinder and its control system. The cylinder is designed to have an optical encoder, pressure sensor, valve and a Programmable System on a Chip (PSoC) as the central processing unit. The PSoC will handle I²C communication, input and output data from the analogue to digital converter, counter program and pulse width modulation (PWM) duty cycle. An application tool for a distributed physical human–machine interaction is proposed using an intelligent pneumatic cylinder. The system applied 36 links of the actuator to form an Intelligent Chair Tool (ICT). The control methodology presented contains an inner force loop and an outer position loop implemented using a unified control system driven by PWM to an on/off valve. In this research, four control approaches, i.e., position control, force control, compliance control and viscosity control, were constructed and experimented. The physical properties of various objects were also detected by the intelligent cylinder through the detecting function experiment. Finally, an emulation experiment using mass was carried out and the results clearly show the ability of the intelligent cylinder, and the control approaches towards realization of the future ICT application.     

Agency Effects in Human–Computer Interaction

The studies presented in this article explore a human-centered conceptualization of agents and agency based on the observation that people attribute agency to sufficiently complex interactive systems. Although agency attribution appears to be an unconscious human response, findings from social psychology, affective computing, and perceptual-motor studies suggest agency attribution influences human–computer interaction (HCI). Three studies are presented that examine whether recent findings on agency attribution in physical environments also apply in the virtual environments characteristic of HCI. Results of the studies indicate that agency effects operate in desktop computing environments. Agency effects, however, appear to be influenced by learning effects that preserve a previously observed relationship between perception and action but alter how this effect is expressed. Results suggest that there are both bottom-up and top-down contributions to agency effects in HCI.     

User Modeling in Human–Computer Interaction

A fundamental objective of human–computer interaction research is to make systems more usable, more useful, and to provide users with experiences fitting their specific background knowledge and objectives. The challenge in an information-rich world is not only to make information available to people at any time, at any place, and in any form, but specifically to say the right thing at the right time in the right way. Designers of collaborative human–computer systems face the formidable task of writing software for millions of users (at design time) while making it work as if it were designed for each individual user (only known at use time). User modeling research has attempted to address these issues. In this article, I will first review the objectives, progress, and unfulfilled hopes that have occurred over the last ten years, and illustrate them with some interesting computational environments and their underlying conceptual frameworks. A special emphasis is given to high-functionality applications and the impact of user modeling to make them more usable, useful, and learnable. Finally, an assessment of the current state of the art followed by some future challenges is given.     

A Safe-Control Paradigm for Human–Robot Interaction

This paper introduces a new approach to control a robot manipulator in a way that is safe for humans in the robot's workspace. Conceptually the robot is viewed as a tool with limited autonomy. The limited perception capabilities of automatic systems prohibits the construction of failsafe robots of the capability of people Instead, the goal of our control paradigm is to make the interaction with a robot manipulator safe by making the robot's actions predictable and understandable to the human operator. At the same time the forces the robot applies with any part of its body to its environment have to be controllable and limited. Experimental results are presented of a human-friendly robot controller that is under development for a Barrett Whole Arm Manipulator robot.     

Remote Tactile Transmission with Time Delay for Robotic Master–Slave Systems

This study develops a method to compensate for the communication time delay for tactile transmission systems. For transmitting tactile information from remote sites, the communication time delay degrades the validity of feedback. However, so far time delay compensation methods for tactile transmissions have yet to be proposed. For visual or force feedback systems, local models of remote environments were adopted for compensating the communication delay. The local models cancel the perceived time delay in sensory feedback signals by synchronizing them with the users' operating movements. The objectives of this study are to extend the idea of the local model to tactile feedback systems and develop a system that delivers tactile roughness of textures from remote environments to the users of the system. The local model for tactile roughness is designed to reproduce the characteristic cutaneous deformations, including vibratory frequencies and amplitudes, similar to those that occur when a human finger scans rough textures. Physical properties in the local model are updated in real-time by a tactile sensor installed on the slave-side robot. Experiments to deliver the perceived roughness of textures were performed using the developed system. The results showed that the developed system can deliver the perceived roughness of textures. When the communication time delay was simulated, it was confirmed that the developed system eliminated the time delay perceived by the operators. This study concludes that the developed local model is effective for remote tactile transmissions.     

Review of Semantic-Free Utterances in Social Human–Robot Interaction

As a young and emerging field in social human–robot interaction (HRI), semantic-free utterances (SFUs) research has been receiving attention over the last decade. SFUs are an auditory interaction means for machines that allow emotion and intent expression, which are composed of vocalizations and sounds without semantic content or language dependence. Currently, SFUs are most commonly utilized in animation movies (e.g., R2-D2, WALL-E, Despicable Me), cartoons (e.g., “Teletubbies,” “Morph,” “La Linea”), and computer games (e.g., The Sims) and hold significant potential for applications in HRI. SFUs are categorized under four general types: Gibberish Speech (GS), Non-Linguistic Utterances (NLUs), Musical Utterances (MU), and Paralinguistic Utterances (PU). By introducing the concept of SFUs and bringing multiple sets of studies in social HRI that have never been analyzed jointly before, this article addresses the need for a comprehensive study of the existing literature for SFUs. It outlines the current grand challenges, open questions, and provides guidelines for future researchers considering to utilize SFU in social HRI.     

A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction

Human visual search plays an important role in many human–computer interaction (HCI) tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as “active vision” (Findlay & Gilchrist, 2003). The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks—and at least partially answers—the four questions of active vision: (a) What can be perceived in a fixation? (b) When do the eyes move? (c) Where do the eyes move? (d) What information is integrated between eye movements? Answers include: (a) Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. (b) The eyes move after the fixated visual stimulus has been processed (i.e., has entered working memory). (c) The eyes tend to go to nearby objects. (d) Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives HCI researchers and practitioners a better understanding of how people visually interact with computers, and provides a theoretical foundation for predictive analysis tools that can predict aspects of that interaction.     

Citizen Science: New Research Challenges for Human–Computer Interaction

Citizen science broadly describes citizen involvement in science. Citizen science has gained significant momentum in recent years, brought about by widespread availability of smartphones and other Internet and communications technologies (ICT) used for collecting and sharing data. Not only are more projects being launched and more members of the public participating, but more human–computer interaction (HCI) researchers are focusing on the design, development, and use of these tools. Together, citizen science and HCI researchers can leverage each other’s skills to speed up science, accelerate learning, and amplify society’s well-being globally as well as locally. The focus of this article is on HCI and biodiversity citizen science as seen primarily through the lens of research in the author’s laboratory. The article is framed around five topics: community, data, technology, design, and a call to save all species, including ourselves. The article ends with a research agenda that focuses on these areas and identifies productive ways for HCI specialists, science researchers, and citizens to collaborate. In a nutshell, while species are disappearing at an alarming rate, citizen scientists who document species’ distributions help to support conservation and educate the public. HCI researchers can empower citizen scientists to dramatically increase what they do and how they do it.     

The Ignorant Led by the Blind: A Hybrid Human–Machine Vision System for Fine-Grained Categorization

We present a visual recognition system for fine-grained visual categorization. The system is composed of a human and a machine working together and combines the complementary strengths of computer vision algorithms and (non-expert) human users. The human users provide two heterogeneous forms of information object part clicks and answers to multiple choice questions. The machine intelligently selects the most informative question to pose to the user in order to identify the object class as quickly as possible. By leveraging computer vision and analyzing the user responses, the overall amount of human effort required, measured in seconds, is minimized. Our formalism shows how to incorporate many different types of computer vision algorithms into a human-in-the-loop framework, including standard multiclass methods, part-based methods, and localized multiclass and attribute methods. We explore our ideas by building a field guide for bird identification. The experimental results demonstrate the strength of combining ignorant humans with poor-sighted machines the hybrid system achieves quick and accurate bird identification on a dataset containing 200 bird species.     

The Design of Human–Machine Interface for Accident Support in Nuclear Power Plants

As feedback from Three Mile Island No. 2, a large amount of human–machine interface (HMI) design has been proposed to support operators during accidents by presenting information on plant status, some of which is implemented in commercial nuclear power plants (NPPs). However, it has not yet been discussed what role HMI should play under conditions where operators must take action as instructed under emergency operating procedure. Regarding this principal issue, an HMI design is proposed together with specific screen images. The advantage of this design is based on the involvement of an ex-operator in NPP in this paper. The participation of users in the design of HMI has been rare. Accordingly the proposed HMI is comprehensive for average operators and is expected to be acceptable for future implementation in commercial NPPs. The effectiveness of the proposed HMI has been examined in a small experiment.     

Stress Detection in Human–Computer Interaction: Fusion of Pupil Dilation and Facial Temperature Features

In order to differentiate the affective state of a computer user as it changes from relaxation to stress, features derived from pupil dilation and periorbital temperature can be utilized. Absolute signal values and measurements computed from these can be fused to increase the accuracy of affective classification. In this study, entropy in a sliding window was used to accommodate the time differences in the physiological rise and fall profiles of pupil and thermal data. Two methods, decision tree and Adaboost with Random Forest (ABRF), were used for classification tests. Detection accuracy of stressful states varied between 65% and 83.8%. Best results can be reported as 83.9% for sensitivity and 83.8% for specificity. ABRF classifier outperformed the decision tree model. This study emphasizes the importance of data fusion, particularly when physiological signals differ with respect to their rise and fall windows across time. Use of entropy within a predefined time window provides a useful set of features to combine with actual measurements. Furthermore, the collection of pupil and thermal data is feasible because surface sensors are eliminated.     

Motion–Display Gain: A New Control–Display Mapping Reflecting Natural Human Pointing Gesture to Enhance Interaction with Large Displays at a Distance

The use of large displays is becoming increasingly prevalent, but development of the usability of three-dimensional (3D) interaction with large displays is still in the early stage. One way to improve the usability of 3D interaction is to develop appropriate control–display (CD) gain function. Nevertheless, unlike in desktop environments, the effects of the relationship between control space and display space in 3D interaction have not been investigated. Moreover, 3D interaction with large displays is natural and intuitive similar to how we work in the physical world. Therefore, a CD gain function that considers human behavior might improve the usability of interaction with large displays. The first experiment was conducted to identify the characteristics of user’s natural hand motion and the user perception of target in distal pointing. Thirty people participated and the characteristics of users’ natural hand movements and the 3D coordinates of their pointing positions were derived. These characteristics were considered in development of motion–display (MD) gain which is a new position-to-position CD mapping. Then, MD gain was experimentally verified by comparing it with Laser pointing, which is currently the best existing CD mapping technique; 30 people participated. MD gain was superior to the existing pointing technique in terms of both performance and subjective satisfaction. MD gain can also be personalized for further improvement. This is an initial attempt to reflect natural human pointing gesture in distal pointing technique, and the developed technique (MD gain) was experimentally proved to be superior to the existing techniques. This achievement is worthy because even a marginal improvement in the performance of pointing task, which is a fundamental and frequent task, can have a large effect on users’ productivity. These results can be used as a resource to understand the characteristics of user’s natural hand movement, and MD gain can be directly applied to situations in which distal pointing is needed, such as interacting with smart TVs or with wall displays. Furthermore, the concept that maps natural human behavior in motor space and an object in visual space can be applied to any interactive system.     

Human–Vehicle Cooperation in Automated Driving: A Multidisciplinary Review and Appraisal

ABSTRACT

To draw a comprehensive and cohesive understanding of human–vehicle cooperation in automated driving, a review is made on key studies in human–robot interaction and human factors. Throughout this article, insight is provided into how human drivers and vehicle systems interplay and influence each other. The limitations of technology-centered taxonomies of automation are discussed and the benefits of accounting for human agents are examined. The contributions of machine learning to automated driving and how critical models in human-system cooperation can inform the design of a more symbiotic relationship between driver and vehicle are investigated. Challenges in the human element to enable the safe introduction of road automation are also discussed. Particularly, the unintended consequences of vehicle automation on driver’s workload, situation awareness and trust are examined, and the social interactions between driver, vehicle, and other road users are investigated. This review will help professionals shape future directions for safer and more efficient and effective human–vehicle cooperation.     

Using GOMS and NASA-TLX to Evaluate Human–Computer Interaction Process in Interactive Segmentation

HCI plays an important role in interactive medical image segmentation. The Goals, Operators, Methods, and Selection rules (GOMS) model and the National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire are different methods that are often used to evaluate the HCI process. In this article, we aim at improving the HCI process of interactive segmentation using both the GOMS model and the NASA-TLX questionnaire to: 1) identify the relations between these two methods and 2) propose HCI design suggestions based on the synthesis of the evaluation results using both methods. For this, we conducted an experiment where three physicians used two interactive segmentation approaches to segment different types of organs at risk for radiotherapy planning. Using the GOMS model, we identified 16 operators and 10 methods. Further analysis discovered strong relations between the use of GOMS operators and the results of the NASA-TLX questionnaire. Finally, HCI design issues were identified, and suggestions were proposed based on the evaluation results and the identified relations.     

Composition of Constraint,Hypothesis and Error Models to improve interaction in Human–Machine Interfaces

Although there are many tasks where output strings are automatically generated from a set of evidence, they are not perfect and human intervention is often required to correct the result. In this paper we present a generic Symbol Input Interaction Method for Human–Machine Interfaces that combines multi-source information: an input Hypothesis Model, an Error Model, a Constraint Model and a user interaction scheme.We use Weighted Finite-State Transducers (WFSTs) to represent the different sources of information available: the initial hypotheses, the possible errors, the constraints imposed by the task (interaction language) and the user input. The fusion of these models to find the most probable output string can be performed efficiently by using carefully selected transducer operations. The proposed system initially suggests an output based on the set of hypotheses, possible errors and Constraint Models. Then, if human intervention is needed, a multimodal approach, where the user input is combined with the aforementioned models, is applied to produce, with a minimum user effort, the desired output. This approach offers the practical advantages of a de-coupled model (e.g. input-system + parameterized rules + post-processor), keeping at the same time the error-recovery power of an integrated approach, where all the steps of the process are performed in the same formal machine (as in a typical HMM in speech recognition) to avoid that an error at a given step remains unrecoverable in the subsequent steps. After a presentation of the theoretical basis of the proposed multi-source information system, its application to two real world problems, as an example of the possibilities of this architecture, is addressed. The experimental results obtained demonstrate that a significant user effort can be saved when using the proposed procedure. A simple demonstration, to better understand and evaluate the proposed system, is available on the web https://demos.iti.upv.es/hi/.     

Implementation of Proteomics Biomarkers in Nephrology: From Animal Models to Human Application?

Preclinical animal models are extensively used in nephrology. In this review, the utility of performing proteome analysis of kidney tissue or urine in such models and transfer of the results to human application has been assessed. Analysis of the literature identified 68 relevant publications. Pathway analysis of the reported proteins clearly indicated links with known biological processes in kidney disease providing validation of the observed changes in the preclinical models. However, although most studies focused on the identification of early markers of kidney disease or prediction of its progression, none of the identified makers has made it to substantial validation in the clinic or at least in human samples. Especially in renal disease where urine is an abundant source of biomarkers of diseases of the kidney and the urinary tract, it therefore appears that the focus should be on human material based discovery studies. In contrast, the most valid information of proteome analysis of preclinical models in nephrology for translation in human disease resides in studies focusing on drug evaluation, both efficacy for translation to the clinic and for mechanistic insight.     

Motivating Human–Agent Interaction: Transferring Insights from Behavioral Marketing to Interface Design

The understanding of consumer interaction with online EC Websites is one of the big current challenges for online marketers. The present paper investigates what drives and impedes Net customers to interact with a marketer's websites and, more specifically, with decision support interface systems (DSISs) employed in these sites to assist users when shopping or searching for high-involvement consumer goods. DSIS are considered as a predecessor or partially already integrator of more elaborated agent systems that may be used in future EC environments. It is shown that the design of today's DSISs is sub-optimal for both marketers and consumers, because it fails to motivate user interaction. One reason for this might be that little effort has been made to transfer the insights from consumer behavior to the design of EC user interfaces. The current paper aims to address this gap by proposing a number of new DSIS design principles, all of which are based on insights from traditional marketing search- and perceived risk theory. The main contribution of this paper is that it proposes a design approach for EC Websites that intuitively makes the user model available to the user [Shneiderman and Maes, 36].