Monitoring children’s developmental progress using augmented toys and activity recognition

Previous research has established the connection between the way in which children interact with objects and the potential early identification of children with autism. Those findings motivate our own work to develop "smart toys," objects embedded with wireless sensors that are safe and enjoyable for very small children, that allow detailed interaction data to be easily recorded. These sensor-enabled toys provide opportunities for autism research by reducing the effort required to collect and analyze a child’s interactions with objects. In the future, such toys may be a useful part of clinical and in-home assessment tools. In this paper, we discuss the design of a collection of smart toys that can be used to automatically characterize the way in which a child is playing. We use statistical models to provide objective, quantitative measures of object play interactions. We also developed a tool to view rich forms of annotated play data for later analysis. We report the results of recognition experiments on more than fifty play sessions conducted with adults and children as well as discuss the opportunities for using this approach to support video annotation and other applications.     

UbiqStack: a taxonomy for a ubiquitous computing software stack

This paper describes a taxonomy for a ubiquitous computing software stack called UbiqStack. Through the lens of the UbiqStack taxonomy we survey a variety of subsystems designed to be the building blocks from which sophisticated infrastructures for ubiquitous computing are assembled. Our experience shows that many of these building blocks fit neatly into one of the five UbiqStack categories, each containing functionally-equivalent components. Effectively identifying the best-fit “Lego pieces”, which in turn determines the composite functionality of the resulting infrastructure, is critical. The selection process, however, is impeded by the lack of convention for labeling these classes of building blocks. The lack of clarity with respect to what ready-made subsystems are available within each class often results in naive re-implementation of ready-made components, monolithic and clumsy implementations, and implementations that impose non-standard interfaces onto the applications above. This paper describes the UbiqStack classes of subsystems and explores each in light of the experience gained over 2 years of active development of both ubiquitous computing applications and software infrastructures for their deployment.     

Prototypes and paratypes: designing mobile and ubiquitous computing applications

Conducting user-centered design is a primary challenge in ubiquitous computing. Two kinds of prototyping techniques can be particularly effective for developing mobile and ubiquitous computing applications: compound prototypes and situated experience prototypes. Compound prototypes combine the final product's user interface (UI) with a computational implementation that runs on a separate system without the resource constraints of the target device. This model lets researchers and designers evaluate a new application's physicality and aesthetics as well as its functionality. Situated experience prototypes, or paratypes, include experimental protocols that attempt to reproduce user interaction with the system in real situations, and can optionally use common prototyping instruments such as paper mock-ups or physical props. Using our Personal Audio Loop (PAL) project as a case study, we discuss how you can fruitfully employ compound prototypes and paratypes when designing and developing mobile and ubiquitous computing applications.     

OOPS: a toolkit supporting mediation techniques for resolving ambiguity in recognition-based interfaces

Recognition technologies are being used extensively in both commercial and research systems. Recognizers are still error-prone however, and this results in performance problems and brittle dialogues, creating barriers to the acceptance and usefulness of recognition systems. Better interfaces to systems using recognition, which can help to reduce the burden of recognition errors, are difficult to build because of lack of knowledge about the ambiguity inherent in recognition. We present a survey of the design of correction techniques in interfaces which make use of recognizers. Based on this survey, we have created a user interface toolkit, OOPS (organized option pruning system) (Mankoff et al., Proceedings of ACM CHI 2000 Conference on Human Factors in Computing Systems, 2000, pp. 368–375). OOPS consists of a library of reusable error correction, or mediation, techniques drawn from the survey, combined with necessary architectural extensions to model and to provide structured support for ambiguity at the input event level of a GUI toolkit. The resulting infrastructure makes it easier for application developers to support error handling, thus helping to reduce the negative effects of recognition errors, and allowing us to explore new types of interfaces for dealing with ambiguity.     

Giving undo attention

The problems associated with the provision of an undo support facility in the context of a synchronous shared or group editor are investigated. Previous work on the development of formal models of ‘undo’ has been restricted to single user systems and has focused on the functionality of undo, as opposed to discussing the support that users require from any error recovery facility. Motivated by new issues that arise in the context of computer supported co-operative work, the authors aim to integrate formal modelling of undo with an analysis of how users understand undo facilities. Together, these combined perspectives of the system and user lead to concrete design advice for implementing an undo facility. The special issues that arise in the context of shared undo also shed light on the emphasis that should be placed on single user undo. In particular, the authors regard undo not as a system command to be implemented, but as a user intention to be supported by the system.     

Viz-A-Vis: toward visualizing video through computer vision

In the established procedural model of information visualization, the first operation is to transform raw data into data tables [1]. The transforms typically include abstractions that aggregate and segment relevant data and are usually defined by a human, user or programmer. The theme of this paper is that for video, data transforms should be supported by low level computer vision. High level reasoning still resides in the human analyst, while part of the low level perception is handled by the computer. To illustrate this approach, we present Viz-A-Vis, an overhead video capture and access system for activity analysis in natural settings over variable periods of time. Overhead video provides rich opportunities for long-term behavioral and occupancy analysis, but it poses considerable challenges. We present initial steps addressing two challenges. First, overhead video generates overwhelmingly large volumes of video impractical to analyze manually. Second, automatic video analysis remains an open problem for computer vision.     

Cyberguide: A mobile context‐aware tour guide

Future computing environments will free the user from the constraints of the desktop. Applications for a mobile environment should take advantage of contextual information, such as position, to offer greater services to the user. In this paper, we present the Cyberguide project, in which we are building prototypes of a mobile context‐aware tour guide. Knowledge of the user's current location, as well as a history of past locations, are used to provide more of the kind of services that we come to expect from a real tour guide. We describe the architecture and features of a variety of Cyberguide prototypes developed for indoor and outdoor use on a number of different hand‐held platforms. We also discuss the general research issues that have emerged in our context‐aware applications development in a mobile environment. This revised version was published online in June 2006 with corrections to the Cover Date.