Continuous sonification enhances adequacy of interactions in peripheral process monitoring

As many users who are charged with process monitoring need to focus mainly on other work while performing monitoring as a secondary task, monitoring systems that purely rely on visual means are often not well suited for this purpose. Sonification, the presentation of data as (non-speech) sound, has proven in several studies that it can help in guiding the user's attention, especially in scenarios where process monitoring is performed in parallel with a different, main task. However, there are several aspects that have not been investigated in this area so far, for example if a continuous soundscape can guide the user's attention better than one that is based on auditory cues. We have developed a system that allows reproducible research to answer such questions. In this system, the participants’ performance both for the main task (simulated by simple arithmetic problems) and for the secondary task (a simulation of a production process) can be measured in a more fine-grained manner than has been the case for existing research in this field. In a within-subject study (n=18), we compared three monitoring conditions – visual only, visual + auditory alerts and a condition combining the visual mode with continuous sonification of process events based on a forest soundscape. Participants showed significantly higher process monitoring performances in the continuous sonification condition, compared to the other two modes. The performance in the main task was at the same time not significantly affected by the continuous sonification.     

Towards an application of interactive sonification for the forces applied on the pedals during cycling on the Wattbike ergometer

Cyclists aim to improve pedaling technique to increase pedal force effectiveness. Therefore, biomechanical feedback about how the force is applied on the pedal stroke throughout the cycle is valuable. This information is typically presented visually on a display mounted on the handlebars. However, challenges associated with visual information presentation motivate movement researchers to display the information acoustically via parameter-mapping sonification that has shown potential for improving motor skills. This paper describes initial considerations for an application of interactive sonification for the forces applied on the pedals during cycling on the Wattbike ergometer. It was aimed to examine if the characteristics of the pedal stroke are represented in the sound to create congruency between sound, action and reaction. The primary focus is on describing the design followed by a brief summary of evaluation and practical effectiveness which was conducted on twenty-four master students (25.9 ± 3.0 years) and tested in a pilot study with four recreational cyclists (35.3 ± 0.9 years).The results of both requests indicate that sonification of the Wattbike data as an acoustic presentation of the force applied throughout the pedal stroke cycle is directly and intuitively understandable. Individual statements from testing revealed that participants became aware of characteristics within the cycling movement which they have not explicitly noticed before. The feeling for specific periods throughout the cycle was improved (pushing and pulling), and the differences in applying force on the legs (symmetry) became evident. Listening to the sound allowed cyclists to hear fluctuations in the forces applied on the pedals and thus try to adapt their muscle activation pattern. Thus, the approach presented here could serve as a helpful training tool for cyclists to optimize their cycling movement.     

Using 3D Sound to Improve the Effectiveness of the Advanced Driver Assistance Systems

Future generation cars will be characterized by a wide range of Information Technology (IT) services providing safety and infotainment. This makes the car an information intensive environment where the visual channel is overloaded, putting the safety of drivers and passengers in jeopardy. We propose the use of a 3D auditory display to provide information from the Advanced Driver Assistance Systems. This reduces the eye-off-road time, exploiting the human capability to associate sounds with positions in space. Preliminary lab tests reveal the suitability of this approach. The system still has to be carefully tuned and personalized to achieve usability and reliability, but we think that it provides a complementary channel that is specially useful in low visibility conditions.     

Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis

Data overload is a generic and tremendously difficult problem that has only grown with each new wave of technological capabilities. As a generic and persistent problem, three observations are in need of explanation: Why is data overload so difficult to address? Why has each wave of technology exacerbated, rather than resolved, data overload? How are people, as adaptive responsible agents in context, able to cope with the challenge of data overload? In this paper, first we examine three different characterisations that have been offered to capture the nature of the data overload problem and how they lead to different proposed solutions. As a result, we propose that (a) data overload is difficult because of the context sensitivity problem – meaning lies, not in data, but in relationships of data to interests and expectations and (b) new waves of technology exacerbate data overload when they ignore or try to finesse context sensitivity. The paper then summarises the mechanisms of human perception and cognition that enable people to focus on the relevant subset of the available data despite the fact that what is interesting depends on context. By focusing attention on the root issues that make data overload a difficult problem and on people’s fundamental competence, we have identified a set of constraints that all potential solutions must meet. Notable among these constraints is the idea that organisation precedes selectivity. These constraints point toward regions of the solution space that have been little explored. In order to place data in context, designers need to display data in a conceptual space that depicts the relationships, events and contrasts that are informative in a field of practice.     

Aware Community Portals: Shared Information Appliances for Transitional Spaces

People wish to maintain a level of awareness of timely information, including presence of others in the workplace and other social settings. We believe this provides better exchange, coordination and contact within a community, especially as people work in asynchronous times and distributed locations. The challenge is to develop lightweight techniques for awareness, interaction and communication using shared information appliances. In this paper, we describe the design of an exploratory responsive display projected within a shared workspace at the MIT Media Lab. The system uses visual sensing to provide relevant information and constructs traces of people’s activity over time. Such aware portals may be deployed in casual workplace domains, distributed workgroups, and everyday public spaces.     

Design of Information Content and Layout for Process Control Based on Goal–Means Domain Analysis

With regard to the design of information content in information display, it is often claimed that the abstraction hierarchy (AH) of the work domain should be considered as a basis for identifying and structuring the information content. The primary advantage of AH-based analysis and design is that functionally abstracted information can systematically be identified and provided to the operator, which has rarely been presented in traditional displays. This study evaluated the effectiveness of providing functional information, which was abstracted and represented based on goal–means analysis along the AH, to the operator in two task situations (fault diagnosis and operation). The results showed that the operator’s performance was improved with the high-level information, and the latter’s utility became greater when the goal–means relations between information at different abstraction levels were exhibited. From the results, three design principles for information display can be drawn. First, information should be identified and displayed at multiple abstraction levels. Second, the goal–means relations among the abstraction levels should be explicitly presented, especially for analytical cognitive tasks. Third, information layout should support information integration along decomposition structure within an abstraction level as well as along abstraction levels.     

Online handwriting recognition: the NPen++ recognizer

This paper presents the online handwriting recognition system NPen++ developed at the University of Karlsruhe and Carnegie Mellon University. The NPen++ recognition engine is based on a multi-state time delay neural network and yields recognition rates from 96% for a 5,000 word dictionary to 93.4% on a 20,000 word dictionary and 91.2% for a 50,000 word dictionary. The proposed tree search and pruning technique reduces the search space considerably without losing too much recognition performance compared to an exhaustive search. This enables the NPen++ recognizer to be run in real-time with large dictionaries. Initial recognition rates for whole sentences are promising and show that the MS-TDNN architecture is suited to recognizing handwritten data ranging from single characters to whole sentences. Received September 3, 2000 / Revised October 9, 2000     

Slow Technology – Designing for Reflection

As computers are increasingly woven into the fabric of everyday life, interaction design may have to change – from creating only fast and efficient tools to be used during a limited time in specific situations, to creating technology that surrounds us and therefore is a part of our activities for long periods of time. We present slow technology: a design agenda for technology aimed at reflection and moments of mental rest rather than efficiency in performance. The aim of this paper is to develop a design philosophy for slow technology, to discuss general design principles and to revisit some basic issues in interaction design from a more philosophical point of view. We discuss examples of soniture and informative art as instances of slow technology and as examples of how the design principles can be applied in practice.     

Using Design Explanation within the Formal Object-Oriented Method

This paper reports the results of an action research project which studied the benefits of documenting the evolution, and the rationale for the evolution, of a requirements specification. The benefits which design explanation offers designers (as documented in the literature) suggested an investigation with a view to understanding the potential contribution of the IBIS (Issue-Based Information System) approach. The paper reports an investigation into the use of ad hoc design explanation, in which design decisions were documented as they were made using the IBIS notation. This study finds both strengths and weaknesses in the approach. It reveals ways in which IBIS might be used more effectively and leads us to suggest a further study into the complementary use of ad hoc and post hoc design explanation approaches.     

An algebra for creating and querying multimedia presentations

Over the last few years, there has been a tremendous increase in the number of interactive multimedia presentations prepared by different individuals and organizations. In this paper, we present an algebra for creating and querying interactive multimedia presentation databases. This algebra operates on trees whose branches reflect different possible playouts of a set of presentations. The algebra not only extends all the classical relational operators to such databases, but also introduces a variety of novel operators for combining multiple presentations. As our algebra supports merging parts or all of existing presentations, this algebra can also be used as an authoring tool for creating multimedia presentations. We prove a host of equivalence results for queries in this algebra, which may be used to build query optimizers for interactive presentation databases.     

Real-time visualisation of fibre networks

Published online: 24 January 2002     

Dynamic playout scheduling algorithms for continuous multimedia streams

In this paper, we investigate a playout scheduling framework for supporting the continuous and synchronized presentations of multimedia streams in a distributed multimedia presentation system. We assume a situation in which the server and network transmissions provide sufficient support for the delivery of media objects. In this context, major issues regarding the enforcement of the smooth presentation of multimedia streams at client sites must be addressed to deal with rate variance of stream presentations and delay variance of networks. We develop various playout-scheduling algorithms that are adaptable to quality-of-service parameters. The proposed algorithms permit the local adjustment of unsynchronized presentations by gradually accelerating or retarding presentation components, rather than abruptly skipping or pausing the presentation materials. A comprehensive experimental analysis of the proposed algorithms demonstrates that our algorithms can effectively avoid playout gaps (or hiccups) in the presentations. This scheduling framework can be readily used to support customized multimedia presentations.     

A data model and a query languagefor multimedia documents databases

A multimedia application involves information that may be in a form of video, images, audio, text and graphics, need to be stored, retrieved and manipulated in large databases. In this paper, we propose an object-oriented database schema that supports multimedia documents and their temporal, spatial and logical structures. We present a document example and show how the schema can adress all the structures described. We also present a multimedia query specification language that can be used to describe a multimedia content portion to be retrieved from the database. The language provides means by which the user can specify the information on the media as well as the temoral and spatial relationships among these media.     

Exploring Context-aware Information Push

Despite much interest over recent years in the area of context-aware computing, there are still a number of significant gaps in our understanding of the HCI issues associated with such systems. One particular issue that remains relatively unexplored is how to design around the apparently conflicting goals of adapting to changes in context while at the same time adhering to the principle of predictability. In this paper, we describe our exploration into this issue through two alternative designs of an interactive context-aware tourist guide. One original design was based around information pull, i.e. the emphasis is on the user to decide when context-aware information is presented. Our second design incorporates the notion of information push whereby the actual presentation of context-aware information is triggered by contextual events, e.g. changes in the user’s location or changes to the opening times of attractions. Through the evaluation of these alternative designs we hope to gain a better understanding of the usability implications relating to push vs. pull in both this specific domain and in interactive context-aware systems in general.     

Dynamic end-to-end QoS management middleware for distributed multimedia systems

In this paper, we present a separable, reusable middleware solution that provides coordinated, end-to-end QoS management over any type of service component, and can use existing (legacy) QoS management solutions (by using wrappers) in a distributed multimedia system. Our middleware solution incorporates strategic and tactical QoS managers, and supports protocols and messages between tactical managers and managed application components, and between QoS managers in the management hierarchy. Strategic QoS managers take a global view of QoS provided by a set of application components within the manager's policy domain. Tactical QoS managers provide local control over application components. We introduce the concept of QoS policy domains to scope the authority of a strategic QoS manager. We describe how the management hierarchy is dynamically configured and reconfigured based on runtime needs of the application.     

Sonification of a network’s self-organized criticality for real-time situational awareness

Communication networks involve the transmission and reception of large volumes of data. Research indicates that network traffic volumes will continue to increase. These traffic volumes will be unprecedented and the behaviour of global information infrastructures when dealing with these data volumes is unknown. It has been shown that complex systems (including computer networks) exhibit self-organized criticality under certain conditions. Given the possibility in such systems of a sudden and spontaneous system reset the development of techniques to inform system administrators of this behaviour could be beneficial. This article focuses on the combination of two dissimilar research concepts, namely sonification (a form of auditory display) and self-organized criticality (SOC). A system is described that sonifies in real time an information infrastructure’s self-organized criticality to alert the network administrators of both normal and abnormal network traffic and operation. It is shown how the system makes changes in a system’s SOC readily perceptible. Implications for how such a system may support real-time situational awareness and post hoc incident analysis are discussed.     

Understanding and Using Context

Context is a poorly used source of information in our computing environments. As a result, we have an impoverished understanding of what context is and how it can be used. In this paper, we provide an operational definition of context and discuss the different ways in which context can be used by context-aware applications. We also present the Context Toolkit, an architecture that supports the building of these context-aware applications. We discuss the features and abstractions in the toolkit that make the task of building applications easier. Finally, we introduce a new abstraction, a situation which we believe will provide additional support to application designers.     

Interactive authoring of multimedia documents in a constraint-based authoring system

As multimedia applications spread widely, it is crucial for programming and design support systems to handle “time” in multimedia documents effectively and flexibly. This paper presents a set of interactive system support tools for designing and maintaining the temporal behavior of multimedia documents. The tool set provides mechanisms for anomaly detection, temporal query processing, and interactive scheduling. It is based on a fast incremental constraint solver we have developed, which can be adapted by any constraint-based system. The incremental constraint solver provides immediate feedback to the user, supporting a highly interactive design process. Combined with existing optimal layout generation mechanisms proposed in the literature, our tools effectively utilize the flexibility provided by constraint-based systems.     

Adaptive multimedia synchronization in a teleconference system

In this paper, we present an adaptive buffering scheme for implementing intra-stream and inter-stream synchronization in real-time multimedia applications. The essence of the proposed scheme is to dynamically enforce equalized delays to incoming media streams, in order to piece-wise smooth the network delay variations and to synchronize the streams at the sink. An adaptive control mechanism based on an event-counting algorithm is employed to calibrate the PlayOut Clocks (POCs), which manages the presentations of multimedia data. The algorithm does not rely on globally synchronized clock and makes minimal assumption on underlying network delay distribution. Also, the user defined quality of service (QoS) specifications can be directly incorporated into the design parameters of the synchronization algorithm. The proposed synchronization scheme has been experimentally implemented in a teleconference system which consists of separately controllable audio, video, and data channels. The modular structure of the synchronization control provides the flexibility to maintain an arbitrary synchronization group in conjunction with a distributed conference management scheme. This paper also shows the experimental results of the test implementation and the suitability of the proposed scheme with respect to the multimedia traffic across an FDDI/Ethernet network.