Modeling the user state for context-aware spoken interaction in ambient assisted living

Ambient Assisted Living (AAL) systems must provide adapted services easily accessible by a wide variety of users. This can only be possible if the communication between the user and the system is carried out through an interface that is simple, rapid, effective, and robust. Natural language interfaces such as dialog systems fulfill these requisites, as they are based on a spoken conversation that resembles human communication. In this paper, we enhance systems interacting in AAL domains by means of incorporating context-aware conversational agents that consider the external context of the interaction and predict the user’s state. The user’s state is built on the basis of their emotional state and intention, and it is recognized by means of a module conceived as an intermediate phase between natural language understanding and dialog management in the architecture of the conversational agent. This prediction, carried out for each user turn in the dialog, makes it possible to adapt the system dynamically to the user’s needs. We have evaluated our proposal developing a context-aware system adapted to patients suffering from chronic pulmonary diseases, and provide a detailed discussion of the positive influence of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.     

Participatory knowledge-management design: A semiotic approach

The aim of this paper is to present a design strategy for collaborative knowledge-management systems based on a semiotic approach. The contents and structure of experts' knowledge is highly dependent on professional or individual practice. Knowledge-management systems that support cooperation between experts from different (sub-)fields need to be situated and tailored to provide effective support even if the common aspects of the data need to be described by ontologies that are generic in respect to the sub-disciplines involved. To understand and approach this design problem, we apply a semiotic perspective to computer application and human–computer interaction. From a semiotic perspective, the computer application is both a message from the designer to the user about the structure of the problem domain, as well as about interaction with it, and a structured channel for the user's communication with herself, himself or other users of the software. Tailoring or “end-user development” – i.e. adapting the knowledge-management system to a specific (sub-)discipline, task or context – then refines both the message and adapts the structure of the interaction to the situated requirements.The essential idea of this paper is to define a new perspective for designing and developing interactive systems to support collaborative knowledge management. The key concept is to involve domain experts in participatory knowledge design for mapping and translating their professional models into the proper vocabularies, notations, and suitable visual structures for navigating among interface elements. To this end, the paper describes how our semiotic approach supports processes for representing, storing, accessing, and transferring knowledge through which the information architecture of an interactive system can be defined. Finally, the results of applying our approach to a real-world case in an archaeological context are presented.     

A Formal Model of Communication and Context Awareness in Multiagent Systems

Awareness is a concept that has been frequently studied in the context of Computer Supported Cooperative Work. However, other fields of computer science can benefit from this concept. Recent research in the multi-agent systems field has highlighted the relevance of complex interaction models such as multi-party communication and context awareness for simulation and adaptive systems. In this article, we present a generic interaction model that enables to use these different models in a standardized way. Emerging as a first-order abstraction, the environment, in the sense of a common medium for the agents, is a suitable paradigm to support the agents’ awareness. We present an operational model, called Environment as Active Support of Interaction, to take into account all the agents that can be interested in a communication. This model is then extended for the regulation of multiagent systems interactions. Priority policies are given to manage the rules governing the context (un-)awareness of the agents. We also present a new AUML connector to create protocols that take into account the agent awareness to implement proactive behaviour, and several communication scenarios are proposed to show practical applications of this model.     

Building a user model implicitly from a cooperative advisory dialog

This paper reviews existing methods for building user models to support adaptive, interactive systems, identifies sigificant problems with these approaches, and describes a new method for implicitly acquiring user models from an ongoing user-system dialog. Existing explicit user model acquisition methods, such as user edited models or model building dialogs put additional burden on the user and introduce artificial model acquisition dialogs. Hand coding stereotypes, another explicit acquisition method, is a tedious and error-prone process. On the other hand, implicit acquisition techniques such as computing presuppositions or entailments either draw too few inferences to be generally useful, or too many to be trusted.In contrast, this paper describes GUMAC, a General User Model Acquisition Component that uses heuristic rules to make default inferences about users' beliefs from their interaction with an advisory expert system. These rules are based on features of human action and conversation that constrain people's behavior and establish expectations about their knowledge. The application of these rules is illustrated with two examples of extended dialogs between users and an investment advisory system. During the course of these conversations, GUMAC is able to acquire an extensive model of the users' beliefs about the aspects of the domain considered in the dialog. These models, in turn, provide the sort of information needed by an explanation generator to tailor explanations the advisory system gives to its users.     

Dialogue based interfaces for universal access

Conversation provides an excellent means of communication for almost all people. Consequently, a conversational interface is an excellent mechanism for allowing people to interact with systems. Conversational systems are an active research area, but a wide range of systems can be developed with current technology. More sophisticated interfaces can take considerable effort, but simple interfaces can be developed quite rapidly. This paper provides an introduction to the current state of the art of conversational systems and interfaces. It describes a methodology for developing conversational interfaces and gives an example of an interface for a state benefits website. The paper discusses how this interface could improve access for a wide range of people and how further development of this interface would allow a larger range of people to use the system with enhanced functionality.     

A general-purpose natural language interface to an operating system

One of the most difficult tasks facing new computer users is the management of data and programs through an operating system (OS) interface. Both command languages and graphical interfaces can impede a user unfamiliar with the terminology and organizational techniques of modern file management systems. Various attempts have been made in recent years to develop systems that will enable users to interact with the computer in natural language. By simplifying the form of user input to an OS, such interfaces allow the user to concentrate on file management tasks rather than on the means of expressing OS-specific instructions. The Informational Network for a Natural Talking (INFANT) System was originally developed as a conversational system for communication at the level of a small child. It has since been adapted for use as a general-purpose English language interface for standard OS file management operations. The INFANT System reduces an input sentence to a hierarchy of standardized propositional forms, which are mapped through a propositional knowledge base and a compositional semantics process into a high-level representation of meaning. Its effectiveness as a working natural language interface has been verified in a pair of experiments performed in a college computer lab. © 1996 John Wiley & Sons, Inc.     

Conversation with and through computers

People design what they say specifically for their conversational partners, and they adapt to their partners over the course of a conversation. A comparison of keyboard conversations involving a simulated computer partner (as in a natural language interface) with those involving a human partner (as in teleconferencing) yielded striking differences and some equally striking similarities. For instance, there were significantly fewer acknowledgments in human/computer dialogue than in human/human. However, regardless of the conversational partner, people expected connectedness across conversational turns. In addition, the style of a partner's response shaped what people subsequently typed. These results suggest some issues that need to be addressed before a natural language computer interface will be able to hold up its end of a conversation.     

计算机串行通信口应用

串行通信作为计算机通信方式之一,主要起到主机与外设以及主机之间的数据传输作用,串行通信具有传输线少、成本低的特点,主要适用于近距离的人-机交换、实时监控等系统通信工作当中,借助于现有的电话网也能实现远距离传输,因此在串行通信接口是计算机系统当中的常用接口。主要在介绍串行通信概况的基础上,介绍串行通信口技术的发展及应用情况。     

Application Study on Visible Human Computer Interaction System of Association Rule

With COM, VB and VC, we develop a visual human computer interaction system, which is used to mine association rules. It can mine association rules from the database which is created by Access and SQL server, as well as the text mode. With the interaction interface, user participates in the process of data mining, making the system mine the satisfying rules.     

Computational modeling of human performance in multiple monitor environments with ACT-R cognitive architecture

Much research on modeling human performance associated with visual perception is formulated by schematic models based on neural mechanisms or cognitive architectures. But, these two modeling paradigms are limited in the domains of multiple monitor environments. Although the schematic model based on neural mechanisms can represent human visual systems in multiple monitor environments by providing a detailed account of eye and head movements, these models cannot easily be applied in complex cognitive interactions. On the other hand, the cognitive architectures can model the interaction of multiple aspects of cognition, but these architectures have not focused on modeling the visual orienting behavior of eye and head movements. Thus, in this study, a specific cognitive architecture, which is ACT-R, is extended by an existing schematic model of human visual systems based on neural mechanisms in order to model human performance in multiple monitor environments more accurately. And, this study proposes a method of modeling human performance using the extended ACT-R. The proposed method is validated by an experiment, confirming that the proposed method is able to predict human performance more accurately in multiple monitor environments.Relevance to industryPredicting human performance with a computational model can be used as an alternative method to implementing iterative user testing for developing a system interface. The computational model in this study can predict human performance in multiple monitor environments, so that the model can be applied early on in the design phase, to evaluate the system interface in multiple monitor environments.     

The challenge of interface design for communication theory: from interaction metaphor to contexts of discovery

Those who study interface design still have not resolved the issue of whether the computer is a passive medium or a communicative participant with which we can hope to ‘engage in conversation’. The authors think that this controversy reflects an inadequate philosophy of interface design, which cannot account for what purpose a medium serves in a human communication. In a short history of humancomputer interface research, we trace the development of this philosophy through three generations of computer interface technology and find that its failure lies in the conceptual limitations of its driving concept of interaction.

Just as the concept of interaction provided the transition from the past (command-line interface) to the present (desktop metaphor interface), perhaps, the concept of engagement can provide the transition from our present understanding of the ‘interactive desktop’ interface to multimedia's ‘contexts of discovery’.

The development of multimedia gives interface designers the ultimate challenge to develop interface technology that will simulate human-to-human communication. Should human communication theory be able to treat the conceptual deficiencies of interface design philosophy? The authors find fundamental challenges here and briefly indicate what Charles S. Peirce's semiotic might offer as an age-old remedy.     

Designing conversational agents: effect of conversational form on our comprehension

We have developed a broadcasting agent system, public opinion channel (POC) caster, which generates understandable conversational form from text-based documents. The POC caster circulates the opinions of community members by using conversational form in a broadcasting system on the Internet. We evaluated its transformation rules in two experiments. In experiment 1, we examined our transformation rules for conversational form in relation to sentence length. Twenty-four participants listened to two types of sentence (long sentences and short sentences) with conversational form or with single speech. In experiment 2, we investigated the relationship between conversational form and the user’s knowledge level. Forty-two participants (21 with a high knowledge level and 21 with a low knowledge level) were selected for a knowledge task and listened to two kinds of sentence (sentences about a well-known topic or sentences about an unfamiliar topic). Our results indicate that the conversational form aided comprehension, especially for long sentences and when users had little knowledge about the topic. We explore possible explanations and implications of these results with regard to human cognition and text comprehension.     

Conversations Over Video Conferences: An Evaluation of the Spoken Aspects of Video-Mediated Communication

Recent trends toward telecommuting, mobile work, and wider distribution of the work force, combined with reduced technology costs, have made video communications more attractive as a means of supporting informal remote interaction. In the past, however, video communications have never gained widespread acceptance. Here we identify possible reasons for this by examining how the spoken characteristics of video-mediated communication differ from face-to-face interaction, for a series of real meetings. We evaluate two wide-area systems. One uses readily available Integrated Services Digital Network (ISDN) lines but suffers the limitations of transmission lags, a half-duplex line, and poor quality video. The other uses optical transmission and video-switching technology with negligible delays, full duplex audio, and broadcast quality video. To analyze the effects of video systems on conversation, we begin with a series of conversational characteristics that have been shown to be important in face-to-face interaction. We identify properties of the communication channel in face-to-face interaction that are necessary to support these characteristics, namely, that it has low transmission lags, it is two way, and it uses multiple modalities. We compare these channel properties with those of the two video-conferencing systems and predict how their different channel properties will affect spoken conversation. As expected, when compared with face-to-face interaction, communication using the ISDN system was found to have longer conversational turns; fewer interruptions, overlaps, and backchannels; and increased formality when switching speakers. Communication over the system with broadcast quality audio and video was more similar to face-to-face meetings, although it did not replicate face-to-face interaction. Contrary to our expectations, formal techniques were still used to achieve speaker switching. We suggest that these may be necessary because of the absence of certain speaker-switching cues. The results imply that the advent of high-speed multimedia networking will improve but not remove all the problems of video conferencing as an interpersonal communications tool, and we describe possible solutions to the outstanding problems.     

Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous

Although avatars may resemble communicative interface agents, they have for the most part not profited from recent research into autonomous embodied conversational systems. In particular, even though avatars function within conversational environments (for example, chat or games), and even though they often resemble humans (with a head, hands, and a body) they are incapable of representing the kinds of knowledge that humans have about how to use the body during communication. Humans, however, do make extensive use of the visual channel for interaction management where many subtle and even involuntary cues are read from stance, gaze, and gesture. We argue that the modeling and animation of such fundamental behavior is crucial for the credibility and effectiveness of the virtual interaction in chat. By treating the avatar as a communicative agent, we propose a method to automate the animation of important communicative behavior, deriving from work in conversation and discourse theory. BodyChat is a system that allows users to communicate via text while their avatars automatically animate attention, salutations, turn taking, back-channel feedback, and facial expression. An evaluation shows that users found an avatar with autonomous conversational behaviors to be more natural than avatars whose behaviors they controlled, and to increase the perceived expressiveness of the conversation. Interestingly, users also felt that avatars with autonomous communicative behaviors provided a greater sense of user control.     

基于WEINVIEW屏和MODBUS的快速展示界面设计及通信软件方案

快速组态化的人机界面开发已经成为当前触摸屏上位机界面展示的趋势,采用标准的工业级通讯协议可以实现多个领域标准化设备接入,本论文以此为应用背景,结合电子产品的设计需求,实现了以WEINVIEW触摸屏为人机交互平台的上位机界面开发和RS232C电气接口下的标准MODBUS RTU通信协议的数据传输的软件系统构架.采用配套组态软件EB8000开发人机交互界面,该软件具有界面友好、快速上手、操作便捷等特点,组态建模资源丰富,大大减少了开发人员界面设计工程量,并且采用MODBUS RTU协议,其帧格式简单、紧凑、使用容易,易与下位机设备对接,满足开发设计人员和用户对产品的需求.本文最后通过实现一种包括上位机界面、通信模块和下位机控制输出的某仪器产品整机软件系统加以辅证.     

Toward Spoken Human–Computer Tutorial Dialogues

Oral discourse is the primary form of human–human communication, hence, computer interfaces that communicate via unstructured spoken dialogues will presumably provide a more efficient, meaningful, and naturalistic interaction experience. Within the context of learning environments, there are theoretical positions supporting a speech facilitation hypothesis that predicts that spoken tutorial dialogues will increase learning more than typed dialogues. We evaluated this hypothesis in an experiment where 24 participants learned computer literacy via a spoken and a typed conversation with AutoTutor, an intelligent tutoring system with conversational dialogues. The results indicated that (a) enhanced content coverage was achieved in the spoken condition; (b) learning gains for both modalities were on par and greater than a no-instruction control; (c) although speech recognition errors were unrelated to learning gains, they were linked to participants' evaluations of the tutor; (d) participants adjusted their conversational styles when speaking compared to typing; (e) semantic and statistical natural language understanding approaches to comprehending learners' responses were more resilient to speech recognition errors than syntactic and symbolic-based approaches; and (f) simulated speech recognition errors had differential impacts on the fidelity of different semantic algorithms. We discuss the impact of our findings on the speech facilitation hypothesis and on human–computer interfaces that support spoken dialogues.     

Designing a goal-oriented smart-home environment

Nowadays, systems are growing in power and in access to more resources and services. This situation makes it necessary to provide user-centered systems that act as intelligent assistants. These systems should be able to interact in a natural way with human users and the environment and also be able to take into account user goals and environment information and changes. In this paper, we present an architecture for the design and development of a goal-oriented, self-adaptive, smart-home environment. With this architecture, users are able to interact with the system by expressing their goals which are translated into a set of agent actions in a way that is transparent to the user. This is especially appropriate for environments where ambient intelligence and automatic control are integrated for the user’s welfare. In order to validate this proposal, we designed a prototype based on the proposed architecture for smart-home scenarios. We also performed a set of experiments that shows how the proposed architecture for human-agent interaction increases the number and quality of user goals achieved.     

FreeWalk: a social interaction platform for group behaviour in a virtual space

FreeWalk is a social interaction platform where people and agents can socially and spatially interact with one another. FreeWalk has evolved to support heterogeneous interaction styles including meetings, cross-cultural encounters, and evacuation drills. Each of them is usually supported by an individual virtual environment. This evolution extended the capability to control social interaction. The first prototype only provides people with an environment in which they can gather to talk with one another while the third prototype provides them with a whole situation to behave according to their assigned roles and tasks. FreeWalk1 is a spatial videoconferencing system. In this system, the positions of participants make spontaneous simultaneous conversations possible. Spatial movements are integrated with video-mediated communication. FreeWalk1 is able to make social interaction more casual and relaxed than telephone-like telecommunication media. In contrast to conventional videoconferencing systems, people formed concurrent multiple groups to greet and chat with others. In FreeWalk2, a social agent acts as an in-between of people to reduce the problem of the low social context in virtual spaces. When the agent notes an awkward pause in a conversation, it approaches those involved in the conversation with a suggestion for a new topic to talk about. We used this agent to support cross-cultural communication between Japan and US. Our agent strongly influenced people's impressions of their partners, and also, their stereotypes about their partner's nationality. FreeWalk3 is a virtual city simulator to conduct virtual evacuation drills. This system brings social interaction into crisis management simulation. People can join a virtual scene of a disaster at home. Social agents can also join to play their roles assigned by simulation designers. The system architecture has a split control interface to divide control of multiple agents into high-level instruction for them and simulation of their low-level actions. The interface helps simulation designers to control many agents efficiently.     

一种噪声和畸变混沌信号的滤波策略-Ⅱ：自适应解调

在这系列文章的第一部分，我们提出了一种状态空间模型的估计算法。第二部分将该算法应用于混沌调制通信系统的解调。当“扮演”系统参数的信息信号被建模为AR模型后，混沌调制通信系统的解调问题可转化为对增广的状态空间模型的估计。仿真结果表明，该算法也能有效地实现混沌调制通信系统的自适应解调。