Designing reality-based interfaces for experiential bio-design

Reality-based interfaces (RBIs) such as tabletop and tangible user interfaces draw upon ideas from embodied cognition to offer a more natural, intuitive, and accessible form of interaction that reduces the mental effort required to learn and operate computational systems. However, to date, little research has been devoted to investigating the strengths and limitations of applying reality-based interaction for promoting learning of complex scientific concepts at the college level. We propose that RBIs offer unique opportunities for enhancing college-level science education. This paper presents three main contributions: (1) design considerations and participatory design process for enhancing college-level science education through reality-based interaction, (2) reflections on the design, implementation, and validation of two case studies—RBIs for learning synthetic biology, and (3) discussion of opportunities and challenges for advancing learning of college-level sciences through next-generation interfaces.     

Exploring how children use their hands to think: an embodied interactional analysis

In order to better understand how to design hands-on child-computer interaction, we explore how different styles of interaction facilitate children's thinking while they use their hands to manipulate objects. We present an exploratory study of children solving a spatial puzzle task. We investigate how the affordances of physical, graphical and tangible interfaces may facilitate the development of thinking skills including mental visualisation, problem space exploration and collaboration. We utilise the theory of complementary actions taken from embodied cognition to develop a video coding methodology that allows us to classify behavioural activity and make inferences about thinking skills development. Our findings indicated that the combination of direct hands-on input style with audio-visual feedback facilitated by the tangible user interface enabled a dynamic task completion strategy, which supports the development of mental skills with a slight time cost. The mouse and graphical user interface supported a trial and error approach, which may limit skills development. The physical cardboard puzzle enabled effective task completion but provided less support for social interaction and problem space exploration. We conclude with design recommendations.     

Two-handed tangible interaction techniques for composing augmented blocks

Modeling tools typically have their own interaction methods for combining virtual objects. For realistic composition in 3D space, many researchers from the fields of virtual and augmented reality have been trying to develop intuitive interactive techniques using novel interfaces. However, many modeling applications require a long learning time for novice users because of unmanageable interfaces. In this paper, we propose two-handed tangible augmented reality interaction techniques that provide an easy-to-learn and natural combination method using simple augmented blocks. We have designed a novel interface called the cubical user interface, which has two tangible cubes that are tracked by marker tracking. Using the interface, we suggest two types of interactions based on familiar metaphors from real object assembly. The first, the screw-driving method, recognizes the user??s rotation gestures and allows them to screw virtual objects together. The second, the block-assembly method, adds objects based on their direction and position relative to predefined structures. We evaluate the proposed methods in detail with a user experiment that compares the different methods.     

A case study of intended versus actual experience of adaptivity in a tangible storytelling system

This article presents a case study of an adaptive, tangible storytelling system called “The Reading Glove”. The research addresses a gap in the field of adaptivity for ubiquitous systems by taking a critical look at the notion of “adaptivity” and how users experience it. The Reading Glove is an interactive storytelling system featuring a wearable, glove-based interface and a set of narratively rich objects. A tabletop display provides adaptive recommendations which highlight objects to select next, functioning as an expert storytelling system. The recommendation engine can be run in three different configurations to examine the effects of different adaptive methods. The study of the design process as well as the user experience of the Reading Glove allows us to develop a deeper understanding of the experience of adaptivity that is useful for designers of intelligent systems, particularly those with ubiquitous and tangible forms of interaction.     

Intra‐organizational user‐centred design practices: The impact of design research departments at design consultancies

The user is a critical factor in design and innovation. Firms experiment with different approaches to involving the user in design processes, which results in new forms of intra‐ and extra‐organizational collaboration. The establishment of in‐house design research units within design consultancies is one such intra‐organizational user‐centred design practice that targets designer‐researcher collaboration. This paper addresses this issue and reports on the findings from multiple case study research exploring the impact of in‐house design research teams on designers' user knowledge construction. We utilized constructivist learning theory to assess major aspects of these intra‐organizational user‐centred design practices. Ethnographically informed field studies were conducted at six design consultancies representing three design fields (i.e., architecture, industrial design and interaction design) in the Northwestern United States. Three of the consultancies have design research departments and three do not. The findings indicate that in‐house design research units play a role in designers' user knowledge construction via their results, processes and human resources. Among these, the active participation of designers in the research process was observed to have the largest impact because of its contribution to designers' contextual and collaborative learning about users.     

User interface development processes and methodologies

The way in which designers choose between alternatives in user interface design can affect both the design process and also the quality of the outcome, i.e., the user interface. However, little is known about the knowledge drawn on during, or the processes that guide, the choice between design alternatives. This paper presents the results of an empirical study aimed at modelling 'trade-off decision making' in user interface design. It is argued that a single abstract vocabulary can capture important aspects of the rich knowledge drawn on during design problem solving. It is also argued that designers' reasoning during 'choice episodes' is not sophisticated. In choosing between alternatives in design, designers invariably limit the range, or nature, of the alternatives considered. The implications of this finding are discussed.     

Bringing tabletop technology to all: evaluating a tangible farm game with kindergarten and special needs children

The tangible interaction approach has in recent years become a promising alternative to multitouch tabletops for very young children. Children learning with tangible user interfaces can benefit from the same pedagogical values as learning with materials in physical play. The NIKVision tabletop and games have been designed to provide leisure and fun while reinforcing physical manipulation and co-located gaming for young children (3- to 6-year-olds). Interaction is provided in NIKVision by the handling of conventional toys and computer augmentation on a table surface. First of all, the paper sets out the results of a summative evaluation carried out in nurseries and schools summarising the performance of the tabletop in terms of usability, user experience and physical and co-located gaming. Secondly, the paper presents an evaluation carried out in a special education school with children with cognitive disabilities, in an attempt to widen the range of possible beneficiaries of tangible interfaces. The challenge in this case is to ensure children’s comprehension of the game and to adequately combine feedback on the application with teacher interventions. In fact, the initial results reinforce the idea of not trying to substitute but rather to assist teachers and emphasise the possibilities offered by the tabletop as a tool to promote student autonomy.     

The influence of a knowledge-based system on designers' cognitive activities: a study involving professional web designers

Over the last few decades, several usability knowledge-based systems have been developed to provide user interface designers with usability knowledge (e.g. heuristics, usability guidelines, standards). Such systems are intended to assist designers during the design process, and to improve the usability of the user interface being designed. However, the assumption that such systems actually improve the usability of the resulting user interface remains to be demonstrated: virtually no systems have been empirically tested by designers who create products. In order to confirm this assumption, we conducted an experimental study in which professional web designers had to create webpages, either using a knowledge-based system – MetroWeb – or without it. This study was intended to determine the influence of MetroWeb on the professional web designers' cognitive activity, and to find out whether MetroWeb actually assists them to develop a user-centred design. The results show that the web designers did not very often use MetroWeb while designing webpages; however, rather surprisingly, the designers who did use MetroWeb more often exhibited a user-centred activity than those working without MetroWeb. We conclude this paper by discussing these findings, and suggesting future possible ways of research intended to assist designers to adopt a user-centred approach to their activity.     

Tangible versus graphical user interfaces for robot programming: exploring cross-age children’s preferences

This study explores children’s opinions and preferences regarding two isomorphic user interfaces that can be used for introductory programming activities, a tangible and a graphical one. The first system (tangible) comprises 46 cube-shaped blocks that represent simple programming structures and can be interconnected to form the programming code. The second system (graphical) presents on-screen the same programming space to the user (icons similar in appearance and operation with the tangible blocks). These two operationally equivalent user interfaces were given to three children groups of different ages (5–6, 7–8 and 11–12 years) to program the behavior of a Lego NXT robot. Children in dyads were let to interact with both systems, and during the activity, data were collected regarding children’s first-sight preference, enjoyment and easiness-to-use. The quantitative and qualitative analysis followed indicated that the tangible interface was more attractive especially for girls, and it was more enjoyable and finally characterized as easier to use only by younger children who were less experienced with computers. On the contrary, for older (11–12 years old) children, the tangible even though was more enjoyable, it was not considered as the easiest-to-use user interface. Taking into account the lack of empirical evidences related to the tangible user interfaces, this study discusses not only the potential usability advantages but also the disadvantages of tangible user interfaces for children.     

The effects of 3D interface metaphor on older adults’ mobile navigation performance and subjective evaluation

Navigating vast information spaces through mobile interfaces has become a common activity in older adults' everyday lives. Studies suggested that interface metaphors could be used to facilitate users' metal model development and information processing when using mobile technologies. However, we know little about how metaphors affect older adults' mobile navigation behavior, and which user characteristics matter during this perceptual and cognitive process. To investigate this, a card interface with a 3D metaphor and a list interface without 3D metaphors were compared among twenty-two participants when performing four navigation tasks. User characteristics including demographic factors, technology experience, and user capabilities were examined. The participants' navigation performance and subjective evaluations were measured as the dependent variables. From the results, we recommend the list interface without 3D metaphors as a beneficial choice for older adults. It performed better in navigation performance, although the differences are not statistically significant. Moreover, navigation performance using the card interface with a 3D metaphor was significantly associated with participants' perceptual speed, thus this interface may be more sensitive to capability declines. Valuable insights into the older adults’ mobile navigation performance and preferences are discussed and important implications for the design of mobile navigation user interfaces are proposed based on the results.Relevance to industryThe experimental results propose a more beneficial way to present contents on a mobile user interface for older adults and provide valuable insights for the designers and industry to help them understand the older adults’ usage and perceptions towards the application of 3D metaphors when navigating with mobile interfaces.     

On tangible user interfaces,humans and spatiality

Like the prehistoric twig and stone, tangible user interfaces (TUIs) are objects manipulated by humans. Tangible user interface success will depend on how well they exploit spatiality, the intuitive spatial skills humans have with the objects they use. In this paper, we carefully examine the relationship between humans and physical objects, and related previous research. From this examination, we distill a set of observations and turn these into heuristics for incorporation of spatiality into TUI application design, a cornerstone for their success. Following this line of thought, we identify spatial TUIs, the subset of TUIs that mediate interaction with shape, space and structure. We then examine several existing spatial TUIs using our heuristics.     

Why not make interfaces better than 3D reality?

Many constrained interfaces are designed to be simpler than the real world by restricting movement, limiting interface actions, and keeping interface objects in a plane. However, the strong utility of pure 3D interfaces for medical, architectural, product design, and scientific visualization means that interface design for pure 3D remains an important challenge. An intriguing possibility is that enhanced 3D interfaces might offer simpler navigation, more compelling functionality, safer movements, and less occlusion, than 3D reality, especially for information exploration and visualization tasks. Such features can enable superhuman capabilities such as faster-than-light teleportation, flying through objects, and X-ray vision. Enhanced 3D interfaces might have supernatural tools such as magic wands for instantly shrinking, enlarging, duplicating, or sending objects and enchanted environments that provide error prevention, history keeping, and programming-by-demonstration. Playful game designers and creative application developers have already pushed the technology further than those who seek merely to mimic reality. Advanced designs are marked by their support of rapid situation awareness through effective overviews, reduced numbers of actions to accomplish tasks; and prompt, meaningful feedback for user actions. This article reviews these clever enhanced 3D-design features and encourages approaches that facilitate user tasks rather than mimic reality.     

Tangible user interfaces for physically-based deformation: design principles and first prototype

We present design principles for conceiving tangible user interfaces for the interactive physically-based deformation of 3D models. Based on these design principles, we developed a first prototype using a passive tangible user interface that embodies the 3D model. By associating an arbitrary reference material with the user interface, we convert the displacements of the user interface into forces required by physically-based deformation models. These forces are then applied to the 3D model made out of any material via a physical deformation model. In this way, we compensate for the absence of direct haptic feedback, which allows us to use a force-driven physically-based deformation model. A user study on simple deformations of various metal beams shows that our prototype is usable for deformation with the user interface embodying the virtual beam. Our first results validate our design principles, plus they have a high educational value for mechanical engineering lectures.     

User modelling for graphical design in complex dynamic environments: concepts and prototype implementations

In this paper, it is proposed that using knowledge-based technology to generate different types of models of a target domain can better assist designers of support systems. The main argument is that designers' decisions should be based on knowledge about a domain rather than on their "common sense" judgments. This approach is illustrated by describing a canonical user model capturing operators' information and knowledge acquisition behaviour. In the present work, this model is used to help designers identify what process variables need to be associated with flow diagrams representing the structure of the technical system. Concepts underlying the user modelling approach as well as the knowledge elicitation and knowledge representation process are described. Finally, functionality provided by the user model is discussed as well as how the approach can be evaluated.     

The Impact of Technology on Creativity in Design: An Enhancement?

Technology may be considered as an interface between individuals and the products they create, but we have to determine whether the use of new systems effectively enhance individuals' creative activities. In this paper, we present a new angle of reflection that we illustrate in the field of creative design, since it is a constant challenge for designers to introduce creativity in the projects they work on. The approach we propose is centred on designers' cognitive processes. We argue that both the development of new CAD (computer‐aided design) systems and their assessment should be conducted on the basis of a deep understanding of designers' cognitive processes. In accordance with this view, we present three empirical studies that were conducted in order to analyse the impact of new design support systems on designers' cognitive processes. Therefore, the results we present contribute to further our knowledge of whether new CAD technologies effectively facilitate designers' activities and enhance their creativity.     

Believable interaction with a quasi‐tangible tabletop interface

In this paper, we present a believable interaction mechanism for manipulation multiple objects in ubiquitous/augmented virtual environment. A believable interaction in multimodal framework is defined as a persistent and consistent process according to contextual experiences and common‐senses on the feedbacks. We present a tabletop interface as a quasi‐tangible framework to provide believable processes. An enhanced tabletop interface is designed to support multimodal environment. As an exemplar task, we applied the concept to fast accessing and manipulating distant objects. A set of enhanced manipulation mechanisms is presented for remote manipulations including inertial widgets, transformable tabletop, and proxies. The proposed method is evaluated in both performance and user acceptability in comparison with previous approaches. The proposed technique uses intuitive hand gestures and provides higher level of believability. It can also support other types of accessing techniques such as browsing and manipulation. Copyright © 2007 John Wiley & Sons, Ltd.     

A Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface

In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user's hand. This paper describes the use of a hand input device for interacting with a 3D graphical application. A dynamic gesture language, which allows users to teach some hand gestures, is presented. Furthermore, a user interface integrating the recognition of these gestures and providing feedback for them, is introduced. Particular attention has been spent on implementing a tool for easy specification of dynamic gestures, and on strategies for providing graphical feedback to users' interactions. To demonstrate that the introduced 3D user interface features, and the way the system presents graphical feedback, are not restricted to a hand input device, a force input device has also been integrated into the user interface.     

MTBuilder:一个多触点交互桌面界面工具

针对基于WIMP范式的图形用户界面工具不能解决多触点交互桌面的多指手势识别、界面组件朝向等问题,提出基于自然用户界面通用隐喻OCGM(objects,containers,gestures and manipulations)的多触点交互桌面界面工具箱——MTBuilder.首先用层次化多触点数据表示模型存储多触点数据,然后对多指手势识别器进行动态管理以加速识别处理,最后基于OCGM设计并实现界面组件库.通过多人信息浏览、城市规划等原型系统的开发和实验评估可以看出,MTBuilder能够为交互桌面界面构造与快速原型系统开发提供强有力的支持.     

Machine learning: A tool to support usability?

One problem facing designers of interactive systems is catering to the wide range of users who will use a particular application. Understanding the user is critical to designing a usable interface. There are a number of ways of addressing this problem, including improved design methodologies using ''intuitive'' interface styles, adaptive interfaces, and better training and user support materials. In this article, we argue that each of these solutions involves pattern recognition in one form or another and that machine learning can therefore aid designers of interactive systems in these areas. We report on experiments that demonstrate the potential of machine learning to user modeling that has application to two of these areas in particular: adaptive systems and design methodologies.     

Artful surfaces: an ethnographic study exploring the use of space in design studios

A largely overlooked aspect of creative design practices is how physical space in design studios plays a role in supporting designers' everyday work. In particular, studio surfaces such as designers' desks, office walls, notice boards, clipboards and drawing boards are full of informative, inspirational and creative artefacts such as, sketches, drawings, posters, story-boards and Post-it notes. Studio surfaces are not just the carriers of information but importantly they are sites of methodic design practices, i.e. they indicate, to an extent, how design is being carried out. This article describes the results of an ethnographic study on the use of workplace surfaces in design studios, from two academic design departments. Using the field study results, the article introduces an idea of ‘artful surfaces’. Artful surfaces emphasise how artfully designers integrate these surfaces into their everyday work and how the organisation of these surfaces comes about helping designers in accomplishing their creative and innovative design practices. Using examples from the field study, the article shows that artful surfaces have both functional and inspirational characteristics. From the field study, three types of artful surfaces are identified: personal; shared; and project-specific. The article suggests that a greater insight into how these artful surfaces are created and used could lead to better design of novel display technologies to support designers' everyday work.