Modeling focus of attention for meeting indexing based on multiple cues

A user's focus of attention plays an important role in human-computer interaction applications, such as a ubiquitous computing environment and intelligent space, where the user's goal and intent have to be continuously monitored. We are interested in modeling people's focus of attention in a meeting situation. We propose to model participants' focus of attention from multiple cues. We have developed a system to estimate participants' focus of attention from gaze directions and sound sources. We employ an omnidirectional camera to simultaneously track participants' faces around a meeting table and use neural networks to estimate their head poses. In addition, we use microphones to detect who is speaking. The system predicts participants' focus of attention from acoustic and visual information separately. The system then combines the output of the audio- and video-based focus of attention predictors. We have evaluated the system using the data from three recorded meetings. The acoustic information has provided 8% relative error reduction on average compared to only using one modality. The focus of attention model can be used as an index for a multimedia meeting record. It can also be used for analyzing a meeting.     

Multiperson visual focus of attention from head pose and meeting contextual cues

This paper introduces a novel contextual model for the recognition of people's visual focus of attention (VFOA) in meetings from audio-visual perceptual cues. More specifically, instead of independently recognizing the VFOA of each meeting participant from his own head pose, we propose to jointly recognize the participants' visual attention in order to introduce context-dependent interaction models that relate to group activity and the social dynamics of communication. Meeting contextual information is represented by the location of people, conversational events identifying floor holding patterns, and a presentation activity variable. By modeling the interactions between the different contexts and their combined and sometimes contradictory impact on the gazing behavior, our model allows us to handle VFOA recognition in difficult task-based meetings involving artifacts, presentations, and moving people. We validated our model through rigorous evaluation on a publicly available and challenging data set of 12 real meetings (5 hours of data). The results demonstrated that the integration of the presentation and conversation dynamical context using our model can lead to significant performance improvements.     

The next-generation business meeting: from i-lands to flexible meeting landscapes

Meetings are an important form of interaction in business settings. However, meetings are often not held in a way that makes all participants feel engaged. Meeting support systems try to overcome this obstacle by helping meeting participants in the meeting workflow. This paper takes a complementary approach by distinguishing between different perspectives of good meetings: the peopleware perspective, the generic groupware perspective, and the roomware perspective. It is argued that all these perspectives have to be considered simultaneously when heading toward an ideal meeting system design. The paper presents excerpts of a pattern language that empowers users to improve their meeting. Besides this, the pattern language can be considered as a tool for the designer. After showing, how both groups made use of the patterns, the paper finally presents LivingAgendas, a meeting support system that can accompany the meeting participant through the meeting life cycle.     

Towards robotic assistants in nursing homes: Challenges and results

This paper describes a mobile robotic assistant, developed to assist elderly individuals with mild cognitive and physical impairments, as well as support nurses in their daily activities. We present three software modules relevant to ensure successful human–robot interaction: an automated reminder system; a people tracking and detection system; and finally a high-level robot controller that performs planning under uncertainty by incorporating knowledge from low-level modules, and selecting appropriate courses of actions. During the course of experiments conducted in an assisted living facility, the robot successfully demonstrated that it could autonomously provide reminders and guidance for elderly residents.     

Personal assistants: Direct manipulation vs. mixed initiative interfaces

Interface agents are computer programmes that provide assistance to users dealing with computer-based applications. The introduction of agents to user interfaces caused the exploration of new metaphors to enhance user ability to directly manipulate interfaces. In this regard, mixed-initiative interaction refers to a flexible interaction strategy in which agents contribute with users by providing suitable information at the most appropriate time. Mixed-initiative approaches promise to dramatically enhance human–computer interaction by allowing agents to resemble human assistants. In this paper, we report a study on how the interaction metaphor can affect the user perception of agent capabilities and, in turn, the final success of agents.     

Engineering computer-based systems: meeting the challenge

Embedded computers in system products have become pervasive. As Engineering of Computer-Based Systems (ECBS) assumes a greater role in computing-product development, engineers face increasing pressure to resolve the challenges of disparate architectures, long interdisciplinary system life cycles, method and tool incompatibility, and spiraling complexity     

Electronic meeting support: the GroupSystems concept

In recent years, there has been a rapidly growing interest in the use of information technology to support face-to-face group meetings. Such Electronic Meeting System (EMS) environments represent a fundamental shift in the technology available for group meetings. In this paper, we describe the development and evaluation research conducted at the University of Arizona that has led to the installation of one EMS at more than 30 corporate and university sites around the world. Based on our experiences in working with student groups in controlled laboratory experiments and with organizational work groups in the field, we are convinced that EMS technology has the potential to dramatically change the way people work together by effectively supporting larger groups, reducing meeting and project time, and enhancing group member satisfaction.     

SmartMic: a smartphone-based meeting support system

Meetings play an important role in our daily lives. Verbal communication and social relationship development are the primary activities of meeting attendees. In this paper, we propose a meeting support system, namely SmartMic, to enhance user experiences and facilitate social interactions during meetings. SmartMic uses smartphone and readily available facilities in common meeting venues to support easy speaking and establishing new social connections. Meeting attendees in the system speak through smartphone instead of microphone and contact with each other conveniently. Preliminary experiment results show that SmartMic can facilitate the social interaction among attendees and promote the meeting productivity efficiently.     

Hotmap: looking at geographic attention

Understanding how people use online maps allows data acquisition teams to concentrate their efforts on the portions of the map that are most seen by users. Online maps represent vast databases, and so it is insufficient to simply look at a list of the most-accessed URLs. Hotmap takes advantage of the design of a mapping system's imagery pyramid to superpose a heatmap of the log files over the original maps. Users' behavior within the system can be observed and interpreted. This paper discusses the imagery acquisition task that motivated Hotmap, and presents several examples of information that Hotmap makes visible. We discuss the design choices behind Hotmap, including logarithmic color schemes; low-saturation background images; and tuning images to explore both infrequently-viewed and frequently-viewed spaces.     

Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds

We conducted an experiment to evaluate the use of embodied survey bots (i.e., software-controlled avatars) as a novel method for automated data collection in 3D virtual worlds. A bot and a human-controlled avatar carried out a survey interview within the virtual world, Second Life, asking participants about their religion. In addition to interviewer agency (bot vs. human), we tested participants’ virtual age, that is, the time passed since the person behind the avatar joined Second Life, as a predictor for response rate and quality. The human interviewer achieved a higher response rate than the bot. Participants with younger avatars were more willing to disclose information about their real life than those with older avatars. Surprisingly, the human interviewer received more negative responses than the bot. Affective reactions of older avatars were also more negative than those of younger avatars. The findings provide support for the utility of bots as virtual research assistants but raise ethical questions that need to be considered carefully.     

Virtual meeting rooms: from observation to simulation

Much working time is spent in meetings and, as a consequence, meetings have become the subject of multidisciplinary research. Virtual Meeting Rooms (VMRs) are 3D virtual replicas of meeting rooms, where various modalities such as speech, gaze, distance, gestures and facial expressions can be controlled. This allows VMRs to be used to improve remote meeting participation, to visualize multimedia data and as an instrument for research into social interaction in meetings. This paper describes how these three uses can be realized in a VMR. We describe the process from observation through annotation to simulation and a model that describes the relations between the annotated features of verbal and non-verbal conversational behavior. As an example of social perception research in the VMR, we describe an experiment to assess human observers’ accuracy for head orientation.

---

Evolving information systems: meeting the ever-changing environment

Abstract. To meet the demands of organizations and their ever-changing environment, information systems are required which are able to evolve to the same extent as organizations do. Such a system has to support changes in all time-and application-dependent aspects. In this paper, requirements and a conceptual framework for evolving information systems are presented. This framework includes an architecture for such systems and a revision of the traditional notion of update. Based on this evolutionary notion of update (recording, correction and forgetting) a state transition-oriented model on three levels of abstraction (event level, recording level, correction level) is introduced. Examples are provided to illustrate the conceptual framework for evolving information systems.     

Graphical displays: implications for divided attention, focused attention, and problem solving.

When completing tasks in complex, dynamic domains observers must consider the relationships among many variables (e.g., integrated tasks) as well as the values of individual variables (e.g., focused tasks). A critical issue in display design is whether or not a single display format can achieve the dual design goals of supporting performance at both types of tasks. We consider this issue from a variety of perspectives. One relevant perspective is the basic research on attention and object perception, which concentrates on the interaction between visual features and processing capabilities. The principles of configurality are discussed, with the conclusion that they support the possibility of achieving the dual design goals. These considerations are necessary but not sufficient for effective display design. Graphic displays map information from a domain into visual features; the tasks to be completed are defined in terms of the domain, not in terms of the visual features alone. The implications of this subtle but extremely important difference are discussed. The laboratory research investigating alternative display formats is reviewed. Much like the attention literature, the results do not rule out the possibility that the dual design goals can be achieved.