Soft computing framework for intelligent human–machine system design,simulation and optimization

 This paper proposes a novel soft-computing framework for human–machine system design and simulation based on the hybrid intelligent system techniques. The complex human–machine system is described by human and machine parameters within a comprehensive model. Based on this model, procedures and algorithms for human–machine system design, economical/ergonomic evaluation, and optimization are discussed in an integrated CAD and soft-computing framework. With a combination of individual neural and fuzzy techniques, the neuro-fuzzy hybrid soft-computing scheme implements a fuzzy if-then rules block for human–machine system design, evaluation and optimization by a trainable neural fuzzy network architecture. For training and test purposes, assembly tasks are simulated and carried out on a self-built multi-adjustable laboratory workstation with a flexible motion measurement and analysis system. The trained neural fuzzy network system is able to predict the operator's postures and joint angles of motion associated with a range of workstation configurations. It can also be used for design/layout and adjustment of human assembly workstations. The developed system provides a unified, intelligent computational framework for human–machine system design and simulation. Case studies for workstation system design and simulation are provided to illustrate and validate the developed system.     

Human Factors in the Design of Supervisory Control Systems and Human–Machine Interfaces for Highly Automated Complex Systems

The design of control systems and human–machine interfaces in the field of complex and safety-critical environments remains today an open issue, in spite of the high technological evolution of the last decades. The increasing use of automation has improved efficiency, safety and ease of operations but, at the same time, it has complicated operators’ situation awareness and has changed the nature of their possible errors. The research activity described in this paper is an attempt to develop a methodological framework to support designers of control systems and human–machine interfaces. In particular, it focuses on the need for a deeply recursive approach related to the implementation of the systemic and human aspects of the design process of a human–machine system, intended as a Joint Cognitive System. A validating case study has been performed, based on the full application of the framework on the control of the turbine/alternator system of a thermoelectric power plant in northern Italy. Correspondence and offprint requests to: M. Piccini, Politecnico di Torino Dipartimento di Energetica, C.so Duca degli Abruzzi 24, 10129 Turin, Italy. Tel.: +39 011 564 4413; Fax: +39 011 564 4499; Email: mipiccin@polito.it     

Sustaining a Community Computing Infrastructure for Online Teacher Professional Development: A Case Study of Designing Tapped In

Community computing has recently grown to become a major research area in human–computer interaction. One of the objectives of community computing is to support computer supported cooperative work among distributed collaborators working toward shared professional goals in online communities of practice. A core issue in designing and developing community computing infrastructures – the underlying socio-technical layer that supports communitarian activities – is sustainability. Many community computing initiatives fail because the underlying infrastructure does not meet end user requirements; the community is unable to maintain a critical mass of users consistently over time; it generates insufficient social capital to support significant contributions by members of the community; or, as typically happens with funded initiatives, financial and human capital resource become unavailable to further maintain the infrastructure. Based on more than nine years of design experience with Tapped In – an online community of practice for education professionals – we present a case study that discusses four design interventions that have sustained the Tapped In infrastructure and its community to date. These interventions represent broader design strategies for developing online environments for professional communities of practice.     

A shared workspace to support man–machine reasoning: application to cooperative distant diagnosis

This paper focuses on man–machine cooperation problems. In particular, it deals with those problems that occur when both human and machine have to achieve a shared reasoning activity. It puts forward a man–machine approach that is dedicated to technical diagnosis problem solving. Coordination of human and automated reasoning is key to solving this problem, since efficiency depends on both sharing and interpreting exchanged data. A shared workspace is proposed to support both machines and their human operators. This workspace is kept as close as possible to human representations in order to reduce cooperation costs. The paper describes those coordination mechanisms that are able to support such a cooperative activity using a shared workspace. In order to assess the costs and benefits of such cooperation, these mechanisms are applied to a complex industrial problem: diagnosis and troubleshooting in a phone network. The results show the full impact of cooperation on human–machine reasoning.     

Representations in Human–Computer Systems Development

When system developers design a computer system (or other information artefact), they must inevitably make judgements as to how to abstract the domain and how to represent this abstraction in their designs. Over the years human–computer interaction, or more generally information systems design, has had a history of developing competing methods and models for both the process and products of its development. Various paradigms have been suggested, often trying to keep pace with the changing nature of the design problem; from batch processing to interactive systems to work situations and most recently to designing for household environments. It appears timely, then, to review the nature of the design problem that faces the developers of human–computer systems and to consider some of the impact that different representations and different conceptualisations may have on their activities. Green (1998) has suggested that a single model of developing human–computer systems is not desirable, instead arguing for a number of limited theories each of which provides a useful perspective. The aim of this paper is to place competing methods side by side in order to see their strengths and weaknesses more clearly. The central tenet of the paper is that different views of both the human–computer system design process and the different abstractions, or models, that are produced during the design process have varying degrees of utility for designers. It is unlikely that any single method or modelling approach will be optimal in all circumstances. Designers need to be aware of the range of views that exist and of the impact that taking a particular approach may have on the design solution.     

The development of an autonomous service robot. Implementation: “Lucas”—The library assistant robot

This paper describes an autonomous mobile device that was designed, developed and implemented as a library assistant robot. A complete autonomous system incorporating human–robot interaction has been developed and implemented within a real world environment. The robotic development is comprehensively described in terms of its localization systems, which incorporates simple image processing techniques fused with odometry and sonar data, which is validated through the use of an extended Kalman filter (EKF). The essential principles required for the development of a successful assistive robot are described and put into demonstration through a human–robot interaction application applied to the library assistant robot.     

A user-centred approach for designing driving support systems: the case of collision avoidance

The work described in this paper is focused on an approach for implementing in real working contexts the guidelines of user-centred design contained in formal standards and in many research studies. The application concerns the EUCLIDE project (enhanced human–machine interface for on vehicle integrated driving support system), which aimed at developing a driving support system to avoid collisions with obstacles in reduced visibility conditions. The design of the system followed a user-centred approach which started by identifying the model of cognition to be applied throughout the whole design process. The definition of the warning strategies of the system was firstly analysed with the aim to achieve the highest balance between a totally supportive system and a non-disturbing system. Then an initial set of design solutions for the human–machine interface was tested in a static driving simulator. A second set of possible interfaces was evaluated in a dynamic simulator before developing a final design. This solution was implemented in two real vehicles and tested in real traffic situations. This paper describes the whole design process and concentrates on the final step of “in-vehicle” integration process. The road tests performed at the end of the whole process are discussed in detail focusing on the safety implications associated with the design solution finally selected and implemented.

The Design of Human–Machine Interface for Accident Support in Nuclear Power Plants

As feedback from Three Mile Island No. 2, a large amount of human–machine interface (HMI) design has been proposed to support operators during accidents by presenting information on plant status, some of which is implemented in commercial nuclear power plants (NPPs). However, it has not yet been discussed what role HMI should play under conditions where operators must take action as instructed under emergency operating procedure. Regarding this principal issue, an HMI design is proposed together with specific screen images. The advantage of this design is based on the involvement of an ex-operator in NPP in this paper. The participation of users in the design of HMI has been rare. Accordingly the proposed HMI is comprehensive for average operators and is expected to be acceptable for future implementation in commercial NPPs. The effectiveness of the proposed HMI has been examined in a small experiment.     

MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination

In this paper we argue that substitution-based function allocation methods (such as MABA-MABA, or Men-Are-Better-At/Machines-Are-Better-At lists) cannot provide progress on human–automation co-ordination. Quantitative ‘who does what’ allocation does not work because the real effects of automation are qualitative: it transforms human practice and forces people to adapt their skills and routines. Rather than re-inventing or refining substitution-based methods, we propose that the more pressing question on human–automation co-ordination is ‘How do we make them get along together?’ Correspondence and offprint requests to: S. W. A. Dekker, Department of Mechanical Engineering, IKP, Link?ping Institute of Technology, SE - 581 83 Link?ping, Sweden. Tel.: +46 13 281646; fax +4613282579; email: sidde@ikp.liu.se     

Experiments in haptic-based authentication of humans

With the rapid advancement of the technological revolution, computer technology such as faster processors, advanced graphic cards, and multi-media systems are becoming more affordable. Haptics technology is a force/tactile feedback technology growing in disciplines linked to human–computer interaction. Similar to the increasing complexity of silicon-based components, haptics technology is becoming more advanced. On the other hand, currently available commercial haptics interfaces are expensive, and their application is mostly dedicated to enormous research projects or systems. However, the trend of the market is forcing haptic developers to release products for use in conjunction with current keyboards and mice technologies. Haptics allows a user to touch, fell, manipulate, create, and/or alter simulated three-dimensional objects in a virtual environment. Most of the existing applications of haptics are dedicated to hone human physical skills such as sensitive hardware repair, medical procedures, handling hazardous substances, etc. These skills can be trained in a realistic virtual world, and describe human behavioural patterns in human–computer interaction environments. The measurement of such psychomotor patterns can be used to verify a person’s identity by assessing unique-to-the-individual behavioural attributes. This paper explores the unique behaviour exhibited by different users interacting with haptic systems. Through several haptic-based applications, users’ physical attributes output data from the haptic interface for use in the construction of a biometric system.

Autonomous Recovery Execution in Nuclear Power Plant by the Agent

This paper has pointed out the necessity of careful decision making by nuclear power plant (NPP) operators based on the critical parameters of an NPP, to maintain safety when these parameters are out of range. Yet under strong time pressure, it is virtually impossible to make optimal decisions in these conditions. The automation of recovery actions may therefore be needed. Considering the requirements for such automation, the paper proposes an autonomous system in collaboration with the human (i.e., an agent system) that will remain effective even during unforeseen conditions. The numerical simulation study showed the effectiveness of the proposed system. The desired relationship between human–machine as the joint system based on a new concept was also proposed.     

Knowledge Networking in Cross-Cultural Settings

Knowledge networking in the cross-cultural setting here focuses on promoting a culture of shared communication, values and knowledge, seeking cooperation through valorisation of diversity. The process is seen here in terms of creating new alliances of creators, users, mediators and facilitators of knowledge. At the global level, knowledge networking is seen as a symbiotic relationship between local and global knowledge resources. This focus is informed by the human-centred vision of the information society, which seeks a symbiotic relationship between technology and society. It explores the nature of the knowledge in transition, raising issues of technology and knowledge transfer in the local–global context. The notions of human–machine symbiosis and of diversity and coherence provide a handle to explore the role of technology for sustainable development. The centrality of knowledge in stimulating knowledge networking for cross-cultural collaboration is illustrated through an exemplar of an EU–India Cross-Cultural Innovation Network project, a collaboration between European and Indian universities and institutes.     

Industrial bank check processing: the A2iA CheckReaderTM

This paper presents the current state of the A2iA CheckReaderTM – a commercial bank check recognition system. The system is designed to process the flow of payment documents associated with the check clearing process: checks themselves, deposit slips, money orders, cash tickets, etc. It processes document images and recognizes document amounts whatever their style and type – cursive, hand- or machine printed – expressed as numerals or as phrases. The system is adapted to read payment documents issued in different English- or French-speaking countries. It is currently in use at more than 100 large sites in five countries and processes daily over 10 million documents. The average read rate at the document level varies from 65 to 85% with a misread rate corresponding to that of a human operator (1%). Received October 13, 2000 / Revised December 4, 2000     

Emotional empathy transition patterns from human brain responses in interactive communication situations

The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain–computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the future for human–machine/environment interaction design. As a test platform for such an intelligent human–machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human–machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments.     

Toward overcoming culture, skill and situation hurdles in Human-Computer Interaction

This paper proposes a new effective strategy for designing and implementing interactive systems overcoming culture, skill and situation hurdles in Human-Computer Interaction (HCI). The strategy to identify and reduce these hurdles is developed in the framework of a methodology based on a recently introduced model of HCI, and exploits the technological innovations of XML (Extensible Markup Language). HCI is modelled as a cyclic process in which the user and the interactive system communicate by materializing and interpreting a sequence of messages. The interaction process is formalized by specifying both the physical message appearance and the computational aspect of the interaction. This formalization allows the adoption of notation traditionally adopted by users in their workplaces as the starting point of the interactive system design. In this way, the human–system interaction language takes into account the users’ culture. Moreover, the methodology permits user representatives to build a hierarchy of systems progressively adapted to users’ situations, skills and habits, according to the work organization in the domain considered. The strategy is proved to be effective by describing how to implement it using BANCO (Browsing Adaptive Network for Changing user Operativity), a feasibility prototype based on XML, which allows the hierarchy implementation and system adaptations. Several examples from an environmental case under study are used throughout the paper to illustrate the methodology and the effectiveness of the technology adopted. Published online: 4 June 2002     

Understanding Perceived Complexity in Human Supervisory Control

Industrial processes are becoming more complex owing to technological developments and new opportunities. Technological developments, hardware and software, have become more reliable and system configurations more robust. However, the reliability of operator control actions has not improved at the same pace. Consequently human reliability has become the relatively weakest aspect of automated, operator-supervised systems. Hence, understanding how the human operator experiences increasing complexity may play an important role in task allocation and human–machine system design. In this paper the perceived complexity is studied within four typical operational environments in supervisory control. Mathematical formulations for these four operational environments are proposed, and their properties are analysed. A laboratory system is used to investigate the perceived complexity under various operational environments. The experimental results show a significantly different perceived complexity for the coupled and uncoupled operation environments. Extrapolation of the results revealed that the operator would have perceived the system as extremely complex if he/she would have to operate more than eight strongly interconnected subsystems extensively in 30 minutes. Implications of this study are also addressed.     

Touch-Space: Mixed Reality Game Space Based on Ubiquitous, Tangible, and Social Computing

This paper presents a novel computer entertainment system which recaptures human touch and physical interaction with the real-world environment as essential elements of the game play, whilst also maintaining the exciting fantasy features of traditional computer entertainment. Our system called ‘Touch-Space’ is an embodied (ubiquitous, tangible, and social) computing based Mixed Reality (MR) game space which regains the physical and social aspects of traditional game play. In this novel game space, the real-world environment is an essential and intrinsic game element, and the human’s physical context influences the game play. It also provides the full spectrum of game interaction experience ranging from the real physical environment (human to human and human to physical world interaction), to augmented reality, to the virtual environment. It allows tangible interactions between players and virtual objects, and collaborations between players in different levels of reality. Thus, the system re-invigorates computer entertainment systems with social human-to-human and human-to-physical touch interactions. Correspondence to: Professor A. Cheok, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260. Email: adriancheok@nus.edu.sg     

User—robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy

This paper describes a hands-off socially assistive therapist robot designed to monitor, assist, encourage, and socially interact with post-stroke users engaged in rehabilitation exercises. We investigate the role of the robot’s personality in the hands-off therapy process, focusing on the relationship between the level of extroversion–introversion of the robot and the user. We also demonstrate a behavior adaptation system capable of adjusting its social interaction parameters (e.g., interaction distances/proxemics, speed, and vocal content) toward customized post-stroke rehabilitation therapy based on the user’s personality traits and task performance. Three validation experiment sets are described. The first maps the user’s extroversion–introversion personality dimension to a spectrum of robot therapy styles that range from challenging to nurturing. The second and the third experiments adjust the personality matching dynamically to adapt the robot’s therapy styles based on user personality and performance. The reported results provide first evidence for user preference for personality matching in the assistive domain and demonstrate how the socially assistive robot’s autonomous behavior adaptation to the user’s personality can result in improved human task performance. This work was supported by USC Women in Science and Engineering (WiSE) Program and the Okawa Foundation.     

To Intervene or not to Intervene: The Dilemma of Management by Exception

Future air traffic management architectures propose to give aircraft more flight path autonomy and turn the air traffic controller into a manager of exceptions. This article reports on one experiment in a series of studies that empirically explored the cognitive work underlying management by exception in air traffic control. Active practitioners (controllers, pilots, dispatchers) were prepared on the rules of the envisioned system and presented with a series of future incidents, each of which they were required to jointly resolve. Management by exception turns out to trap human controllers in a double bind, where intervening early seems appealing but is difficult to justify (airspace throughput) and carry out (controller workload problems). Late interventions are just as difficult, since controllers will have to take over in the middle of a potentially challenging or deteriorating situation. Computerised decision support that flags exceptions migrates the decision criterion into a device, creating a threshold crossing that is typically set either too early or too late. This article lays out the intertwined trade-offs and dilemmas for the exception manager, and makes recommendations for cooperative human–machine architectures in future air traffic management.     

PessimalPrint: a reverse Turing test

Abstract. We exploit the gap in ability between human and machine vision systems to craft a family of automatic challenges that tell human and machine users apart via graphical interfaces including Internet browsers. Turing proposed [Tur50] a method whereby human judges might validate “artificial intelligence” by failing to distinguish between human and machine interlocutors. Stimulated by the “chat room problem” posed by Udi Manber of Yahoo!, and influenced by the CAPTCHA project [BAL00] of Manuel Blum et al. of Carnegie-Mellon Univ., we propose a variant of the Turing test using pessimal print: that is, low-quality images of machine-printed text synthesized pseudo-randomly over certain ranges of words, typefaces, and image degradations. We show experimentally that judicious choice of these ranges can ensure that the images are legible to human readers but illegible to several of the best present-day optical character recognition (OCR) machines. Our approach is motivated by a decade of research on performance evaluation of OCR machines [RJN96,RNN99] and on quantitative stochastic models of document image quality [Bai92,Kan96]. The slow pace of evolution of OCR and other species of machine vision over many decades [NS96,Pav00] suggests that pessimal print will defy automated attack for many years. Applications include `bot' barriers and database rationing. Received: February 14, 2002 / Accepted: March 28, 2002 An expanded version of: A.L. Coates, H.S. Baird, R.J. Fateman (2001) Pessimal Print: a reverse Turing Test. In: {\it Proc. 6th Int. Conf. on Document Analysis and Recognition}, Seattle, Wash., USA, September 10–13, pp. 1154–1158 Correspondence to: H. S. Baird