Making it easier for older people to talk to smart homes: the effect of early help prompts

It is well known that help prompts shape how users talk to spoken dialogue systems. This study investigated the effect of help prompt placement on older users’ interaction with a smart home interface. In the dynamic help condition, help was only given in response to system errors; in the inherent help condition, it was also given at the start of each task. Fifteen older and sixteen younger users interacted with a smart home system using two different scenarios. Each scenario consisted of several tasks. The linguistic style users employed to communicate with the system (interaction style) was measured using the ratio of commands to the overall utterance length (keyword ratio) and the percentage of content words in the user’s utterance that could be understood by the system (shared vocabulary). While the timing of help prompts did not affect the interaction style of younger users, it was early task-specific help supported older users in adapting their interaction style to the system’s capabilities. Well-placed help prompts can significantly increase the usability of spoken dialogue systems for older people.     

Modeling sequences of user actions for statistical goal recognition

User goals are of major importance for an interface agent because they serve as a context to define what the user’s focus of attention is at a given moment. The user’s goals should be detected as soon as possible, after observing few user actions, in order to provide the user with timely assistance. In this article, we describe an approach for modeling and recognizing user goals from observed sequences of user actions by using Variable Order Markov models combined with an exponential moving average (EMA) on the prediction probabilities. The validity of our approach has been tested using data collected from real users in the Unix domain. The results obtained show that an interface agent can achieve near 90% average accuracy and over 58% online accuracy in predicting the most probable user goal after each observed action, in a time linear to the number of goals being modeled. We also found that the use of an EMA allows a faster convergence in the actual user goal.     

Language and Space: a two-level semantic approach based on principles of ontological engineering

An increasing number of applications for dialogue systems presuppose an ability to deal appropriately with space. Dialogues with assistance systems, intelligent mobility devices and navigation systems all commonly involve the use of spatial language. For smooth interaction, this spatial language cannot be interpreted ‘in the abstract’—it must instead be related directly to a user’s physical location, orientation, goals and needs and be embedded appropriately in a system’s interaction. This is far from straightforward. The situated interpretation of natural language concerning space, spatial relationships and spatial activities represents an unsolved challenge at this time. Despite extensive work on spatial language involving many disciplines, there are no generally accepted accounts that provide support for the kind of flexible language use observed in real human-human spatial dialogues. In this paper, I review some recent approaches to the semantics for natural language expressions concerning space in order to motivate a two-level semantic-based approach to the interpretation of spatial language. This draws on a new combination of natural language processing and principles of ontological engineering and stands as a foundation for more sophisticated and natural dialogue system behavior where spatial information is involved.     

Modeling Multimodal Expression of User’s Affective Subjective Experience

With the growing importance of information technology in our everyday life, new types of applications are appearing that require the understanding of information in a broad sense. Information that includes affective and subjective content plays a major role not only in an individual’s cognitive processes but also in an individual’s interaction with others. We identify three key points to be considered when developing systems that capture affective information: embodiment (experiencing physical reality), dynamics (mapping experience and emotional state with its label) and adaptive interaction (conveying emotive response, responding to a recognized emotional state). We present two computational systems that implement those principles: MOUE (Model Of User Emotions) is an emotion recognition system that recognizes the user’s emotion from his/her facial expressions, and from it, adaptively builds semantic definitions of emotion concepts using the user’s feedback; MIKE (Multimedia Interactive Environment for Kansei communication) is an interactive adaptive system that, along with the user, co-evolves a language for communicating over subjective impressions.     

Toward accessible self-service kiosks through intelligent user interfaces

Public self-service kiosks provide key services such as ticket sales, airport check-in and general information. Such kiosks must be universally designed to be used by society at large, irrespective of the individual users’ physical and cognitive abilities, level of education and familiarity with the system. The noble goal of universal accessibility is hard to achieve. This study reports experiences with a universally designed kiosk prototype based on a multimodal intelligent user interface that adapts to the user’s physical characteristics. The user interacts with the system via a tall rectangular touch-sensitive display where the interaction area is adjusted to fit the user’s height. A digital camera is used to measure the user’s approximate reading distance from the display such that the text size can be adjusted accordingly. The user’s touch target accuracy is measured, and the target sizes are increased for users with motor difficulties. A Byzantine visualization technique is employed to exploit unused and unreachable screen real estate to provide the user with additional visual cues. The techniques explored in this study have potential for most public self-service kiosks.     

A Probabilistic Approach for Argument Interpretation

We describe a probabilistic approach for the interpretation of user arguments, and investigate the incorporation of different models of a user’s beliefs and inferences into this mechanism. Our approach is based on the tenet that the interpretation intended by the user is that with the highest posterior probability. This approach is implemented in a computer-based detective game, where the user explores a virtual scenario, and constructs an argument for a suspect’s guilt or innocence. Our system receives as input an argument entered through a web interface, and produces an interpretation in terms of its underlying knowledge representation – a Bayesian network. This interpretation may differ from the user’s argument in its structure and in its beliefs in the argument propositions. We conducted a synthetic evaluation of the basic interpretation mechanism, and a user-based evaluation which assesses the impact of the different user models. The results of both evaluations were encouraging, with the system generally producing argument interpretations our users found acceptable. The revised version of this article was published in July 2005 with corrections to Table II.     

Adaptation of robot’s perception of fuzzy linguistic information by evaluating vocal cues for controlling a robot manipulator

This article proposes a method for adapting a robot’s perception of fuzzy linguistic information by evaluating vocal cues. The robot’s perception of fuzzy linguistic information such as “very little” depends on the environmental arrangements and the user’s expectations. Therefore, the robot’s perception of the corresponding environment is modified by acquiring the user’s perception through vocal cues. Fuzzy linguistic information related to primitive movements is evaluated by a behavior evaluation network (BEN). A vocal cue evaluation system (VCES) is used to evaluate the vocal cues for modifying the BEN. The user’s satisfactory level for the robot’s movements and the user’s willingness to change the robot’s perception are identified based on a series of vocal cues to improve the adaptation process. A situation of cooperative rearrangement of the user’s working space is used to illustrate the proposed system by a PA-10 robot manipulator.     

Interactive volumetric information visualization for document corpus management

This paper describes aminimally immersive three-dimensional volumetric interactive information visualization system for management and analysis of document corpora. The system, SFA, uses glyph-based volume rendering, enabling more complex data relationships and information attributes to be visualized than traditional 2D and surface-based visualization systems. Two-handed interaction using three-space magnetic trackers and stereoscopic viewing are combined to produce aminimally immersive interactive system that enhances the user’s three-dimensional perception of the information space. This new system capitalizes on the human visual system’s pre-attentive learning capabilities to quickly analyze the displayed information. SFA is integrated with adocument management and information retrieval engine named Telltale. Together, these systems integrate visualization and document analysis technologies to solve the problem of analyzing large document corpora. We describe the usefulness of this system for the analysis and visualization of document similarity within acorpus of textual documents, and present an example exploring authorship of ancient Biblical texts. Received: 15 December 1997 / Revised: June 1999     

User—robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy

This paper describes a hands-off socially assistive therapist robot designed to monitor, assist, encourage, and socially interact with post-stroke users engaged in rehabilitation exercises. We investigate the role of the robot’s personality in the hands-off therapy process, focusing on the relationship between the level of extroversion–introversion of the robot and the user. We also demonstrate a behavior adaptation system capable of adjusting its social interaction parameters (e.g., interaction distances/proxemics, speed, and vocal content) toward customized post-stroke rehabilitation therapy based on the user’s personality traits and task performance. Three validation experiment sets are described. The first maps the user’s extroversion–introversion personality dimension to a spectrum of robot therapy styles that range from challenging to nurturing. The second and the third experiments adjust the personality matching dynamically to adapt the robot’s therapy styles based on user personality and performance. The reported results provide first evidence for user preference for personality matching in the assistive domain and demonstrate how the socially assistive robot’s autonomous behavior adaptation to the user’s personality can result in improved human task performance. This work was supported by USC Women in Science and Engineering (WiSE) Program and the Okawa Foundation.     

A multi-agent based user context Bayesian neural network analysis system

The increasing user mobility demands placed upon IT services necessitates an environment that enables users to access optimal services at any time and in any place. This study presents research conducted to develop a system that is capable of analyzing user IT service patterns and tendencies and provides the necessary service resources by sharing each user’s context information. First, each user’s context information is gathered to provide the multi-agent software training data necessary to describe user operations in a hybrid peer-to-peer (P2P) structured communication environment. Next, the data collected about each user’s mobile device is analyzed through a Bayesian based neural network system to identify the user’s tendency and extract essential service information. This information provides a communication configuration allowing the user access to the best communication service between the user’s mobile device and the local server at any time and in any place, thereby enhancing the ubiquitous computing environment.     

Discovering frequent user--environment interactions in intelligent environments

Intelligent Environments are expected to act proactively, anticipating the user’s needs and preferences. To do that, the environment must somehow obtain knowledge of those need and preferences, but unlike current computing systems, in Intelligent Environments, the user ideally should be released from the burden of providing information or programming any device as much as possible. Therefore, automated learning of a user’s most common behaviors becomes an important step towards allowing an environment to provide highly personalized services. In this article, we present a system that takes information collected by sensors as a starting point and then discovers frequent relationships between actions carried out by the user. The algorithm developed to discover such patterns is supported by a language to represent those patterns and a system of interaction that provides the user the option to fine tune their preferences in a natural way, just by speaking to the system.     

Understanding user commands by evaluating fuzzy linguistic information based on visual attention

This article proposes a method for understanding user commands based on visual attention. Normally, fuzzy linguistic terms such as “very little” are commonly included in voice commands. Therefore, a robot’s capacity to understand such information is vital for effective human-robot interaction. However, the quantitative meaning of such information strongly depends on the spatial arrangement of the surrounding environment. Therefore, a visual attention system (VAS) is introduced to evaluate fuzzy linguistic information based on the environmental conditions. It is assumed that the corresponding distance value for a particular fuzzy linguistic command depends on the spatial arrangement of the surrounding objects. Therefore, a fuzzy-logic-based voice command evaluation system (VCES) is proposed to assess the uncertain information in user commands based on the average distance to the surrounding objects. A situation of object manipulation to rearrange the user’s working space is simulated to illustrate the system. This is demonstrated with a PA-10 robot manipulator.     

Towards dynamic behavior-based profiling for reducing spatial information overload in map browsing activity

The quantity of available detailed spatial content over the Web is continually growing. This leads to the problems of information overload and lengthy map download and render times. In order to address these problems in an effective and unobtrusive manner, the available content must be implicitly filtered and prioritized according to the user’s interests. Personalization of the user’s map elides less relevant information and prioritizes relevant information. The authors previously introduced a novel technique for detailed behavior-based spatial profiling. This article provides an analysis of the technique while exploring the properties of user interactions with a typical Web-based map browsing system. A technique for the automatic identification of specific interaction patterns is introduced and explored in a bid to improve current behavior-based map personalization techniques. The goal of this work is to move towards real-time profiling to support spatial dataset personalization, thus improving the user experience by reducing information overload.     

Information enquiry kiosk with multimodal user interface

A multimodal interactive dialogue automaton (kiosk) for self-service is presented in the paper. Multimodal user interface allow people to interact with the kiosk by natural speech, gestures additionally to the standard input and output devices. Architecture of the kiosk contains key modules of speech processing and computer vision. An array of four microphones is applied for far-field capturing and recording of user’s speech commands, it allows the kiosk to detect voice activity, to localize sources of desired speech signals, and to eliminate environmental acoustical noises. A noise robust speaker-independent recognition system is applied to automatic interpretation and understanding of continuous Russian speech. The distant speech recognizer uses grammar of voice queries as well as garbage and silence models to improve recognition accuracy. Pair of portable video-cameras are applied for vision-based detection and tracking of user’s head and body position inside of the working area. Russian-speaking talking head serves both for bimodal audio-visual speech synthesis and for improvement of communication intelligibility by turning the head to an approaching client. Dialogue manager controls the flow of dialogue and synchronizes sub-modules for input modalities fusion and output modalities fission. The experiments made with the multimodal kiosk were directed to cognitive and usability studies of human-computer interaction by different communication means     

A capabilities-based model for adaptive organizations

Multiagent systems have become popular over the last few years for building complex, adaptive systems in a distributed, heterogeneous setting. Multiagent systems tend to be more robust and, in many cases, more efficient than single monolithic applications. However, unpredictable application environments make multiagent systems susceptible to individual failures that can significantly reduce its ability to accomplish its overall goal. The problem is that multiagent systems are typically designed to work within a limited set of configurations. Even when the system possesses the resources and computational power to accomplish its goal, it may be constrained by its own structure and knowledge of its member’s capabilities. To overcome these problems, we are developing a framework that allows the system to design its own organization at runtime. This paper presents a key component of that framework, a metamodel for multiagent organizations named the Organization Model for Adaptive Computational Systems. This model defines the requisite knowledge of a system’s organizational structure and capabilities that will allow it to reorganize at runtime and enable it to achieve its goals effectively in the face of a changing environment and its agent’s capabilities.     

Better Living Through Geometry

Mark Weiser described ubiquitous computing as “invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere”[1]. The EasyLiving project is concerned with development of an architecture and technologies for ubiquitous computing environments that allow the dynamic aggregation of diverse I/O devices into a single coherent user experience. Though the need for research in distributed computing, perception and interfaces is widely recognised, the importance of an explicit geometric world model for enhancing the user’s experience of a ubiquitous computing system has not been well-articulated. This paper describes three scenarios that benefit from geometric context and introduces the EasyLiving Geometric Model.     

Method to analyze user behavior in home environment

The development of computing technology affects the environmental change of residential area a lot in a many ways. In other words, we believe that Ubiquitous computing technology will change our very vision of our homes. The most distinct phenomenon of Ubiquitous environment is that the number of personal computers will increase mainfold. That means in the near future we will coexist with computers in routine life and people will have more computers to manage. In terms of User Interface, the interaction between systems and users in home-network environment should step up the next level which is different from present method of using computer. Therefore, Ubiquitous environment should be embodied for people and it must be started with understanding users and grasping sophisticated and fundamental needs of them. In this research study, we have attempted to give shape to ‘Method to Analyze User Behavior’ to make embodiment of user interface in Ubiquitous environment by considering and analyzing user’s behavior patterns. With this aim, we have extracted contexts at home from a case study and will suggest analyzing method by classifying data of the study.     

Towards personality-based user adaptation: psychologically informed stylistic language generation

Conversation is an essential component of social behavior, one of the primary means by which humans express intentions, beliefs, emotions, attitudes and personality. Thus the development of systems to support natural conversational interaction has been a long term research goal. In natural conversation, humans adapt to one another across many levels of utterance production via processes variously described as linguistic style matching, entrainment, alignment, audience design, and accommodation. A number of recent studies strongly suggest that dialogue systems that adapted to the user in a similar way would be more effective. However, a major research challenge in this area is the ability to dynamically generate user-adaptive utterance variations. As part of a personality-based user adaptation framework, this article describes personage, a highly parameterizable generator which provides a large number of parameters to support adaptation to a user’s linguistic style. We show how we can systematically apply results from psycholinguistic studies that document the linguistic reflexes of personality, in order to develop models to control personage’s parameters, and produce utterances matching particular personality profiles. When we evaluate these outputs with human judges, the results indicate that humans perceive the personality of system utterances in the way that the system intended.     

Finding User’s Interest Blocks using Significant Implicit Evidence for Web Browsing on Small Screen Devices

Recent researches on improving the efficiency and user experience of Web browsing on handhelds are seeking to solve the problem by re-authoring Web pages or making adaptations and recommendations according to user preference. Their basis is a good understanding of the relationship between user behaviors and user preference. We propose a practical method to find user’s interest blocks by machine learning using the combination of significant implicit evidences, which is extracted from four aspects of user behaviors: display time, viewing information items, scrolling and link selection. We also develop a customized Web browser for small screen devices to collect user behaviors accurately. For evaluation, we conduct an on-line user study and make statistical analysis based on the dataset, which shows that most types of the suggested implicit evidences are significant, and viewing information items is the least indicative aspect of user behaviors. The dataset is then processed off-line to find user’s interest blocks using the proposed method. Experimental results demonstrate the effectiveness of finding user’s interest blocks by machine learning using the combination of significant implicit evidences. Further analysis reveals the great effect of users and moderate effect of Websites on the usefulness of significant implicit evidences.