Drag-and-drop multimedia: ¶an interface framework for digital libraries

In this paper, we describe a new interface for querying multimedia digital libraries and an interface building framework. The interface employs a drag-and-drop style of interaction and combines a structured natural-language style query specification with reusable multimedia objects. We call this interface DanDMM, short for “drag-and-drop multimedia”. DanDMM interfaces capture the syntax of the underlying query language, and dynamically reconfigure to reflect the contents of the data repository. A distinguishing feature of DanDMM is its ability to synthesize integrated interfaces that incorporate both example-based specification using multimedia objects, and traditional techniques including keyword, attribute, and free text-based search. We describe the DanDMM-builder, a framework for synthesizing DanDMM interfaces, and give several examples of interfaces that have been constructed using DanDMM-builder, including a remote-sensing library application and a video digital library. Received: 15 December 1997 / Revised: June 1999     

Shared virtual worlds for education: the ExploreNet experiment

ExploreNet is an experimental environment for creating and delivering networked “virtual worlds.” This system's style of user interaction was inspired by the concept of a “habitat” as first articulated in the LucasFilm's Habitat system. Players enter and interact in a habitat via their animated alter egos, called “avatars.” Habitats may be created for many purposes, including social interaction, entertainment and education. Our focus has been to facilitate the creation of habitats in which virtual communities of learners and mentors interact. This paper presents details of the current ExploreNet system, including its user interface, the means it provides for creating complex behaviors, details of its implementation, the outcomes of several experiments using this system, and our plans for its natural migration to a World Wide Web-based system.     

What are a Location’s “File” and “Edit” Menus?

The promise of mobile devices lies not in their capacity to duplicate the capabilities of desktop machines, but rather in their promise of enabling location-specific tasks. One of the challenges that must be addressed if they are to be used in this way is how intuitive interfaces for mobile devices can be designed that enable access to location-specific services usable across locations. We are developing a prototype mobile valet application that presents location-specific services organised around the tasks associated with a location. The basic elements of the interface exploits commonalties in the way we address tasks at various locations just as the familiar “file” and “edit” menus in various software applications exploit regularities in software tasks.     

A novel soft computing architecture for the control of autonomous walking robots

An integration of concepts from neurobiology, applied psychology, insect physiology and behaviour based robotics has led us to propose a novel generic systems architecture for the intelligent control of mobile robots and in particular, autonomous walking machines. (We define what we mean by “autonomy”.) The control architecture is hierarchical and will be described from a top-down perspective. Level one consists of interpreting a motivation and translating this into high-level commands. Once a high-level command is generated, a range of internal representations or “cognitive maps” may be employed at level two to help provide body-centred motion. At level three of the hierarchy kinematic planning is performed. The fourth level – dynamic compensation – requires feedback from the actuators and compensates for errors in the target vectors provided by the kinematic level and caused by systematic dynamic uncertainties or environmental disturbances. This is implemented using adaptive neural controllers. The interfaces will be described and results from simulation and implementation of levels 2–4 on a hexapod robot will be presented. The hierarchy employs the following soft computing techniques: evolution strategies, cognitive maps, adaptive heuristic critics, temporal difference learning and adaptive neural control using linear-equivalent neural networks.     

Transporting QoS adaptive flows

Distributed audio and video applications need to adapt to fluctuations in delivered quality of service (QoS). By trading off temporal and spatial quality to available bandwidth, or manipulating the playout time of continuous media in response to variation in delay, audio and video flows can be made to adapt to fluctuating QoS with minimal perceptual distortion. In this paper, we extend our previous work on a QoS Architecture (QoS-A) by populating the QoS management planes of our architecture with a framework for the control and management of multilayer coded flows operating in heterogeneous multimedia networking environments. Two key techniques are proposed: i) an end-to-end rate-shaping scheme which adapts the rate of MPEG-coded flows to the available network resources while minimizing the distortion observed at the receiver; and ii) an adaptive network service, which offers “hard” guarantees to the base layer of multilayer coded flows and “fairness” guarantees to the enhancement layers based on a bandwidth allocation technique called Weighted Fair Sharing.     

Design and analysis of a video-on-demand server

The availability of high-speed networks, fast computers and improved storage technology is stimulating interest in the development of video on-demand services that provide facilities similar to a video cassette player (VCP). In this paper, we present a design of a video-on-demand (VOD) server, capable of supporting a large number of video requests with complete functionality of a remote control (as used in VCPs), for each request. In the proposed design, we have used an interleaved storage method with constrained allocation of video and audio blocks on the disk to provide continuous retrieval. Our storage scheme interleaves a movie with itself (while satisfying the constraints on video and audio block allocation. This approach minimizes the starting delay and the buffer requirement at the user end, while ensuring a jitter-free display for every request. In order to minimize the starting delay and to support more non-concurrent requests, we have proposed the use of multiple disks for the same movie. Since a disk needs to hold only one movie, an array of inexpensive disks can be used, which reduces the overall cost of the proposed system. A scheme supported by our disk storage method to provide all the functions of a remote control such as “fast-forwarding”, “rewinding” (with play “on” or “off”), “pause” and “play” has also been discussed. This scheme handles a user request independent of others and satisfies it without degrading the quality of service to other users. The server design presented in this paper achieves the multiple goals of high disk utilization, global buffer optimization, cost-effectiveness and high-quality service to the users.     

Specifying and Analysing System-Level Inter-Component Interfaces

In control systems, the interfaces between software and its embedding environment are a major source of costly errors. For example, Lutz reported that 20–35% of the safety-related errors discovered during integration and system testing of two spacecraft were related to the interfaces between the software and the embedding hardware. Also, the software’s operating environment is likely to change over time, further complicating the issues related to system-level inter-component communication. In this paper we discuss a formal approach to the specification and analysis of inter-component communication using a revised version of RSML (Requirements State Machine Language). The formalism allows rigorous specification of the physical aspects of the inter-component communication and forces encapsulation of communication-related properties in well-defined and easy-to-read interface specifications. This enables us both to analyse a system design to detect incompatibilities between connected components and to use the interface specifications as safety kernels to enforce safety constraints.     

InfoPoint: A Device that Provides a Uniform User Interface to Allow Appliances to Work Together over a Network

This paper proposes a new hand-held device called “InfoPoint” that allows appliances to work together over a network. We have applied the idea of “drag-and-drop” operation as provided in the GUIs of PC and workstation desktop environment. InfoPoint provides a unified interface that gives different types of appliances “drag-and-drop”-like behaviour for the transfer of data. Moreover, it can transfer data from/to non-appliances such as pieces of paper. As a result, InfoPoint allows appliances to work together, in the real-world environment, in terms of data transfer. A prototype of InfoPoint has been implemented and several experimental applications have been investigated. InfoPoint has shown its applicability in a variety of circumstances. We believe that the idea proposed in this paper will be a significant technology in the network of the future.     

Managing the verification trajectory

In this paper we take a closer look at the automated analysis of designs, in particular of verification by model checking. Model checking tools are increasingly being used for the verification of real-life systems in an industrial context. In addition to ongoing research aimed at curbing the complexity of dealing with the inherent state space explosion problem – which allows us to apply these techniques to ever larger systems – attention must now also be paid to the methodology of model checking, to decide how to use these techniques to their best advantage. Model checking “in the large” causes a substantial proliferation of interrelated models and model checking sessions that must be carefully managed in order to control the overall verification process. We show that in order to do this well both notational and tool support are required. We discuss the use of software configuration management techniques and tools to manage and control the verification trajectory. We present Xspin/Project, an extension to Xspin, which automatically controls and manages the validation trajectory when using the model checker Spin. Published online: 18 June 2002     

User interaction in digital libraries: coping with diversity through adaptation

User interface adaptations can be used to address several user interaction challenges in the development of digital library systems. This paper: (a) examines some of the intrinsic characteristics of digital library systems; (b) identifies some of the key Human–Computer Interaction (HCI) challenges in relation to design and architectural abstraction for user interfaces to digital library systems; and (c) develops an argumentation for adaptations in digital library systems. By drawing parallels to recent HCI research into adaptable and adaptive user interaction, the paper illustrates potential areas in which user interface adaptation can provide a useful technique for advancing the quality of human interaction with a digital library system. Received: 18 December 1998 / Revised: 31 May 1999     

PessimalPrint: a reverse Turing test

Abstract. We exploit the gap in ability between human and machine vision systems to craft a family of automatic challenges that tell human and machine users apart via graphical interfaces including Internet browsers. Turing proposed [Tur50] a method whereby human judges might validate “artificial intelligence” by failing to distinguish between human and machine interlocutors. Stimulated by the “chat room problem” posed by Udi Manber of Yahoo!, and influenced by the CAPTCHA project [BAL00] of Manuel Blum et al. of Carnegie-Mellon Univ., we propose a variant of the Turing test using pessimal print: that is, low-quality images of machine-printed text synthesized pseudo-randomly over certain ranges of words, typefaces, and image degradations. We show experimentally that judicious choice of these ranges can ensure that the images are legible to human readers but illegible to several of the best present-day optical character recognition (OCR) machines. Our approach is motivated by a decade of research on performance evaluation of OCR machines [RJN96,RNN99] and on quantitative stochastic models of document image quality [Bai92,Kan96]. The slow pace of evolution of OCR and other species of machine vision over many decades [NS96,Pav00] suggests that pessimal print will defy automated attack for many years. Applications include `bot' barriers and database rationing. Received: February 14, 2002 / Accepted: March 28, 2002 An expanded version of: A.L. Coates, H.S. Baird, R.J. Fateman (2001) Pessimal Print: a reverse Turing Test. In: {\it Proc. 6th Int. Conf. on Document Analysis and Recognition}, Seattle, Wash., USA, September 10–13, pp. 1154–1158 Correspondence to: H. S. Baird     

The use of cursor measures for motion-impaired computer users

“Point and click” interactions remain one of the key features of graphical user interfaces (GUIs). People with motion-impairments, however, can often have difficulty with accurate control of standard pointing devices. This paper discusses work that aims to reveal the nature of these difficulties through analyses that consider the cursor’s path of movement. A range of cursor measures was applied, and a number of them were found to be significant in capturing the differences between able-bodied users and motion-impaired users, as well as the differences between a haptic force feedback condition and a control condition. The cursor measures found in the literature, however, do not make up a comprehensive list, but provide a starting point for analysing cursor movements more completely. Six new cursor characteristics for motion-impaired users are introduced to capture aspects of cursor movement different from those already proposed. Published online: 6 November 2002     

Mechanized result verification: an industrial application

We present part of an industrial project where mechanized theorem proving is used for the validation of a translator which generates safety critical software. In this project, the mechanized proof is decomposed in two parts: one is done “online”, at each run of the translator, by a custom prover which checks automatically that the result of each translation meets some verification conditions; the other is done “offline”, once for all, interactively with a general purpose prover; the offline proof shows that the verification conditions checked by the online prover are sufficient to guarantee the correctness of each translation. The provably correct verification conditions can thus be seen as specifications for the online prover. This approach is called mechanized result verification. This paper describes the project requirements and explains the motivations to formal validation by mechanized result verification, provides an overview of the formalization of the specifications for the online prover and discusses in detail some issues we have addressed in the mechanized offline proof.     

Simplicity in cognitive assistive technology: a framework and agenda for research

Technology offers substantial benefits to the many people with some form of cognitive disability. But the power of technology often comes in a package whose complexity is a barrier to many users, leading to calls for designs, and especially designs for user interfaces, that are “simple”. This paper analyzes the idea of simplicity, and suggests (a) that simplicity in a user interface is not a unified concept, but rather has distinguishable facets, and (b) that simplicity must be defined in terms of the cognitive capabilities of a user, so that what is “simpler” for one user may be “more complex” for another. Despite (b), the prospects for universal design in this area are good, in that interface technology with the flexibility needed to produce “simple” interfaces for a range of users with different cognitive strengths will be of value in addressing the overall design space of interfaces for a broad audience. While it is possible to sketch the outlines of a useful theory of simplicity, the sketch reveals much that is not fully understood. It also reveals opportunities to rethink the architecture of user interfaces in a way that will benefit user interface development generally.     

A case study in class library verification: Java’s vector class

This paper presents a verification of an invariant property for the Vector class from JAVA’s standard library (API). The property says (essentially) that the actual size of a vector is less than or equal to its capacity. It is shown that this “safety” or “data integrity” property is maintained by all methods of the Vector class, and that it holds for all objects created by the constructors of the Vector class. The verification of the Vector class invariant is done with the proof tool PVS. It is based on a semantics of JAVA in higher order logic. The latter is incorporated in a special purpose compiler, the LOOP tool, which translates JAVA classes into logical theories. It has been applied to the Vector class for this case study. The actual verification takes into account the object-oriented character of JAVA: (non-final) methods may always be overridden, so that one cannot rely on a particular implementation. Instead, one has to reason from method specifications in such cases. This project demonstrates the feasibility of tool-assisted verification of non-trivial properties for non-trivial JAVA classes. Published online: 10 May 2001     

Restoring subsampled color images

In some capturing devices, such as digital cameras, there is only one color sensor at each pixel. Usually, 50% of the pixels have only a green sensor, 25% only a red sensor, and 25% only a blue sensor. The problem is then to restore the two missing colors at each pixel – this is called “demosaicing”, because the original samples are usually arranged in a mosaic pattern. In this short paper, a few demosaicing algorithms are developed and compared. They all incorporate a notion of “smoothness in chroma space”, by imposing conditions not only on the behavior of each color channel separately, but also on the correlation between the three channels.     

Display Design of Process Systems Based on Functional Modelling

The prevalent way to present information in industrial computer displays is by using piping and instrumentation diagrams. Such interfaces have sometimes resulted in difficulties for operators because they are not sufficient to fulfil their needs. A systematic way that supports interface design therefore has to be considered. In the new design framework, two questions must be answered. Firstly, a modelling method is required to describe a process system. Such a modelling method can define the information content that must be displayed in interfaces. Secondly, how to communicate this information to operators efficiently must be considered. This will provide a basis for determining the visual forms that the information should take. This study discusses interface design of human–machine systems from these two points of view. Based on other scholars’ work, a comprehensive set of functional primitives is summarised as a basis to build a functional model of process systems. A library of geometrical presentations for these primitives is then developed. To support effective interface design, the concept of ‘functional macro’ is introduced and a way to map functional model to interface display is illustrated by applying several principles. To make our ideas clear, a central heating system is taken as an example and its functional model is constructed. Based on the functional model, the information to be displayed is determined. Several functional macros are then found in the model and their corresponding displays are constructed. Finally, by using the library of geometrical presentations for functional primitives and functional macros, the display hierarchy of the central heating system is developed. Reusability of functional primitives makes it possible to use the methodology to support interface design of different process systems.     

Edges: saliency measures and automatic thresholding

Edges are useful features for structural image analysis, but the output of standard edge detectors must be thresholded to remove the many spurious edges. This paper describes experiments with both new and old techniques for: 1. determining edge saliency (as alternatives to gradient magnitude) and 2. automatically determining appropriate edge threshold values. Some examples of edge saliency measures are lifetime, wiggliness, spatial width, and phase congruency. Examples of thresholding techniques use: the Rayleigh distribution to model the edge gradient magnitude histogram, relaxation labelling, and an edge curve “length”–“average gradient magnitude” feature space.     

Motion of disturbances: detection and tracking of multi-body non-rigid motion

We present a new approach to the tracking of very non-rigid patterns of motion, such as water flowing down a stream. The algorithm is based on a “disturbance map”, which is obtained by linearly subtracting the temporal average of the previous frames from the new frame. Every local motion creates a disturbance having the form of a wave, with a “head” at the present position of the motion and a historical “tail” that indicates the previous locations of that motion. These disturbances serve as loci of attraction for “tracking particles” that are scattered throughout the image. The algorithm is very fast and can be performed in real time. We provide excellent tracking results on various complex sequences, using both stabilized and moving cameras, showing a busy ant column, waterfalls, rapids and flowing streams, shoppers in a mall, and cars in a traffic intersection. Received: 24 June 1997 / Accepted: 30 July 1998     

Interface-Me: Pursuing Sociability Through Personal Devices

In this paper we describe the fundamental principles that guide our work process at 5050 Ltd, in developing concepts and prototypes for personal technology devices. We maintain that in designing personal devices it is critical to address the social interaction elements of the user experience. We introduce the term “social functionality” to refer to those aspects of a device that are specifically designed to elicit sociability and serendipity. It also refers to those aspects which enable users to communicate or represent individuating characteristics through the use of their device. Social functionality is seen as a critical success factor in the design of future personal devices. Introducing social functionality in personal technology devices requires a multidisciplinary approach. Design and technology are seen as inseparable elements of the development process. The mbracelet¹, a wearable prototype we developed for, and in association with, NCR’s The Knowledge Lab, is used as a case in point.