Towards the design of usable multimodal interaction languages

This paper presents novel recommendations for the design of usable multimodal command or query languages. These recommendations have been inferred from the results of three empirical studies focused on the use of spontaneous speech (first study) and the synergic use of spontaneous versus controlled speech and gestures for interacting with current application software (second and third studies). In particular, we propose a method for designing multimodal languages that can be considered as an appropriate substitute for direct manipulation in all contexts precluding the use of mouse and keyboard, and for all standard categories of users, especially the general public.     

Supporting the use of guidelines and style guides in professional user interface design

The use of user-interface design knowledge such as general guidelines and environment-specific style guides can be valuable, and is increasingly required in professional user-interface design. However, conventional guidelines and style guides ( and ) in the form of documents are consistently found hard to use. We have earlier demonstrated knowledge-based critiquing to be a technically feasible way of delivering relevant and knowledge. The purpose of this study was to investigate the need and acceptability of such techniques for professional user-interface designers.

An experiment was carried out where four professional designers developed user-interface prototypes to a functional specification. The designs were evaluated using our and -based critiquing system, which identified a total of 17 deviations from style-guide requirements or design recommendations. Interviews were conducted with the designers to find the reasons for the deviations and to identify important requirements for a critiquing design-support tool. The deviation analysis points to an existing need for better ways of accessing and knowledge. The interviews indicate that the designers would find a critiquing tool valuable, provided that it leaves them in control of their work and indicates the severity of the detected deviations.     

面向盲人的图形显示设计方法及其用户体验研究

针对已有的盲人触觉显示器所展示的图像不能有效地为视觉残障人士提供准确的信息的问题,从盲人对电子触觉图像的需求出发,通过分析盲人在使用传统触觉图像中所遇问题以及涉及到的人机因素的相关背景知识,结合新型显示方案的需求,总结并提出盲人触觉图像设计准则.通过实验对比传统图像与根据该准则重新设计的T-图像,实验结果表明,T-图像...     

Designing for radical tailorability: coupling artifact and rationale

Creativity and cognition are discussed in the context of tailoring computer applications. Computer applications, such as word processors and graphics editors, suffer from a lack of informal design rationale to accompany and throw light upon the meaning behind the implementation of the application. The working hypothesis is that coupling uninterpreted rationale (such as pictures, diagrams, stories, scenarios, and argumentation) with formal software artifacts will give professional computer users enough background understanding for them to be able to make creative and meaningful extensions to the foreground artifacts with only a partial understanding of the implementation language.     

飞行模拟系统人机交互设计

针对增强飞行模拟训练的沉浸感,提高飞行模拟训练的效率和效果的目的,以真实飞行训练为基础,利用UI分类和设计方法,对飞行训练关键交互操作以及核心人机交互UI进行研究并加以整合,运用虚拟仿真技术,着力解决飞行模拟训练中人机交互体验沉浸感低,人性化设计不足,交互方式单一,训练效率和效果不佳等问题。提出将飞行模拟训练系统人机交互UI分为沙盒人机交互UI模块,平显、仪表人机交互UI模块,头戴式显示器人机交互UI模块,样机驾驶舱人机交互环境模块四个部分,完成了四个模块的设计,并基于Unigine引擎实现了四个模块。有效提升了飞行模拟训练的效率和训练效果。     

Adaptively shortened pull down menus: location knowledge and selection efficiency

Adaptively shortened pull down menus, as introduced by Microsoft with the ‘personalized menus’ in Office 2000? and assumed to speed up menu selection, are examined. Displacement of items in this adaptively changing menu is argued to conflict with the user's location knowledge, leading to increased selection times and error rates. In a controlled experiment with 31 subjects, a standard menu was compared with a shortened and a gapped menu variant. The gapped menu variant served to distinguish effects from displacement and effects resulting from reduced item number. Selection times and error rates were smaller for the standard menu than for the shortened variant. The gapped menu, whilst it has longer distances, was faster than the shortened menu, but slower than the standard menu. According to our findings, the assumption that adaptively shortened pull down menus facilitate menu selection is weakened.     

Computer-based task representation: a methodology for improving system design

Task analysis procedures can be used in all stages of system development to examine the nature and severity of performance demands placed upon human operators. Task analyses involve considerable effort in obtaining a detailed set of measurements that represent the behavioural and cognitive tasks performed by the operators; these measures are referred to as task representation. The present article describes a flexible method for incorporating task representation data into an electronic spreadsheet. Such a computer-based procedure has many advantages over the more traditional paper and pencil approaches to task representation and analysis, including greater flexibility in data analysis and the inclusion of both objective and subjective measures of workload. Portions of an illustrative task representation and task analysis are presented along with a discussion of the strengths of computer-based task representation and analysis.     

人机交互的进展及面临的挑战

叙述了人机交互的最近进展,尤其是我国的一些优秀成果,包括多通道交互、交互模型、虚拟现实和三维交互、可穿戴计算和移动计算、智能空间、标准化;并论述了无所不在计算等对人机交互提出的挑战．最后发表了若干看法。     

The challenge of interface design for communication theory: from interaction metaphor to contexts of discovery

Those who study interface design still have not resolved the issue of whether the computer is a passive medium or a communicative participant with which we can hope to ‘engage in conversation’. The authors think that this controversy reflects an inadequate philosophy of interface design, which cannot account for what purpose a medium serves in a human communication. In a short history of humancomputer interface research, we trace the development of this philosophy through three generations of computer interface technology and find that its failure lies in the conceptual limitations of its driving concept of interaction.

Just as the concept of interaction provided the transition from the past (command-line interface) to the present (desktop metaphor interface), perhaps, the concept of engagement can provide the transition from our present understanding of the ‘interactive desktop’ interface to multimedia's ‘contexts of discovery’.

The development of multimedia gives interface designers the ultimate challenge to develop interface technology that will simulate human-to-human communication. Should human communication theory be able to treat the conceptual deficiencies of interface design philosophy? The authors find fundamental challenges here and briefly indicate what Charles S. Peirce's semiotic might offer as an age-old remedy.     

User Interfaces and Help Systems: From Helplessness to Intelligent Assistance

Despite a large body of multidisciplinary research on helpful and user-orientedinterface design, help facilities found in most commercial software are so ill-conceived thatthey are often unhelpful. From a wide spectrum of disciplines and software tools, we presentan extensive review of related work, identifying their limitations as well as their most prom-isingaspects. Using this material, we attempt to recapitulate the necessary requirements foruseful help systems.     

面向大屏幕目标选择的眼动跟踪与手势交互

眼动跟踪方法具有很强的视觉指向性,可以将其应用于面向大屏幕的目标选择,进而避免鼠标操作方式在空间上的远距离移动.然而,仅仅利用眼动跟踪进行选择操作,也会产生选择精度降低、容易产生误操作等问题.因此,为了实现大屏幕上快速、准确的目标选择,提出一种融合眼动跟踪与手势的多通道交互方法,即通过眼动跟踪选择目标,利用手势进行选择确认.在目标尺寸小、目标间距较小时,通过光标稳定和二次选择机制进一步对交互过程进行优化.用户测试结果表明,该方法可以在大屏幕上针对不同尺寸和间距的目标完成有效的选择操作,与仅使用眼动跟踪的目标选择方法相比,任务完成速度提升了16%,任务完成正确率提升了82.6%.此外,针对层级菜单的具体选择任务,该方法与仅使用眼动跟踪的方法相比,任务完成速度提升了13.6%,任务完成正确率提升了55.7%.此外,该方法总体性能接近传统的鼠标操作方式,进一步验证了该方法在实际应用中的有效性.     

Designing peer-to-peer distributed user interfaces: Case studies on building distributed applications

Building a distributed user interface (DUI) application should ideally not require any additional effort beyond that necessary to build a non-distributed interface. In practice, however, DUI development is fraught with several technical challenges such as synchronization, resource management, and data transfer. In this paper, we present three case studies on building distributed user interface applications: a distributed media player for multiple displays and controls, a collaborative search system integrating a tabletop and mobile devices, and a multiplayer Tetris game for multi-surface use. While there exist several possible network architectures for such applications, our particular approach focuses on peer-to-peer (P2P) architectures. This focus leads to a number of challenges and opportunities. Drawing from these studies, we derive general challenges for P2P DUI development in terms of design, architecture, and implementation. We conclude with some general guidelines for practical DUI application development using peer-to-peer architectures.     

Design and evaluation of an adaptive icon toolbar

As information systems become increasingly important in many different domains, the potential to adapt them to individual users and their needs also becomes more important. Adaptive user interfaces offer many possible ways to adjust displays and improve procedures for a user's individual patterns of work. This paper describes an attempt to design an adaptive user interface in a computer environment familiar to many users. According to one classification of adaptive user interfaces, the adaptive bar described in this paper would be classified as a user-controlled self-adaptation system.At the user's convenience, the adaptive bar offers suggestions for adding or removing command icons, based on the frequency and probability of specific commands. It also implements these changes once the user has agreed to them. Beyond the adaptive bar, the general behavior of the whole user interface does not change, thereby allowing the user to maintain a clear general model of the system. This paper describes the decision-making algorithm implemented in the bar. It also describes the bar's self-adaptive behavior of displaying the frequency of each icon's use through the icon's size. Finally, we present some encouraging preliminary results of evaluations by users.     

Towards useful and usable interaction design tools: CanonSketch

Despite all the effort dedicated to bringing better User-Centered Design (UCD) tools to market, current studies show that the industry is still dominated by tools that do not support the activities and workstyles of designers. Also, there is a growing need for interaction design tools aimed at software engineers, a problem related to bringing usability into the software engineering processes.

We propose a new workstyle model that can be effectively used to envision, design and evaluate a new generation of innovative interaction and software design tools, aimed at integrating usability and software engineering.

We illustrate the effectiveness of our model by describing a new tool, called CanonSketch, that was built in order to support UCD in terms of the dimensions in our workstyle model. We also describe an evaluation study aimed at contrasting paper prototyping with our tool as well as the level of workstyle support.     

一种时序数据的可视化技术与工具

时序数据在许多研究领域中都广泛存在.信息可视化是对时序数据进行分析的有力工具,然而已有的时序数据可视化技术无法较好地分析时间跨度长的时序数据.提出一种称为FisheyeLines的可视化技术,能够快速显示大数据集的整体视图,同时方便查看细节信息;能够清楚地表达复杂信息间的相互关系,同时可以显示对象的属性.基于FisheyeLines可视化技术,还给出了一种称为FisheyeLinesVis的可视化工具,可以方便地构造时序数据可视化应用.对以上技术和工具所构造的应用实例的分析与评估充分验证了该方法的有效性和易用性.     

Intelligent Assistance in Authoring Dynamically Generated Web Interfaces

Since its emergence in the early 1990s, the WWW has become not only an information system of unprecedented size, but a universal platform for the development of services and applications. However, most of the advances in web technologies are intended for professional developers, paying poor attention to end-users with no programming abilities but with explicit needs of creating and customizing web-based presentations. This provides a strong motivation for end-users to act as designers at some point, leading to an emerging role of new computing-related professionals to be considered. This paper is an effort to leverage such difficulties by providing intelligent mechanism to assist end-users in web-based authoring tasks. To carry out such a challenge, intelligent user-monitoring techniques are exploited to obtain high-level information that will be used to infer the user’s preferences and assist him throughout the interaction. Furthermore, we report on how iteration patterns can be applied to avoid repetitive tasks that are automatically carried out on behalf of the user. In order to bring off a feasible trade-off between expressivity and ease of use, a user experiment to obtain the user’s perception and evaluate the hit-rate of our system is also presented.     

Intelligent interaction design: the role of human-computer interaction research in the design of intelligent systems

As more intelligent systems are introduced into the marketplace, it is becoming increasingly urgent to consider usability for such systems. Historically, the two fields of artificial intelligence (AI) and human- computer interaction (HCI) have had little in common. In this paper, we consider how established HCI techniques can usefully be applied to the design and evaluation of intelligent systems, and where there is an urgent need for new approaches. Some techniques - notably those for requirements acquisition and empirical evaluation - can usefully be adopted, and indeed are, within many projects. However, many of the tools and techniques developed within HCI to support design and theory-based evaluation cannot be applied in their present forms to intelligent systems because they are based on inappropriate assumptions; there is consequently a need for new approaches. Conversely, there are approaches that have been developed within AI - e.g. in research on dialogue and on ontologies - that could usefully be adapted and encapsulated to respond to this need. These should form the core of a future research agenda for intelligent interaction design.     

Designing for pen and speech input in an object-action framework: the case of email

This study presents a user interface that was intentionally designed to support multimodal interaction by compensating for the weaknesses of speech compared with pen input and vice versa. The test application was email using a web pad with pen and speech input. In the case of pen input, information was represented as visual objects, which were easily accessible. Graphical metaphors were used to enable faster and easier manipulation of data. Speech input was facilitated by displaying the system speech vocabulary to the user. All commands and accessible fields with text labels could be spoken in by name. Commands and objects that the user could access via speech input were shown on a dynamic basis in a window. Multimodal interaction was further enhanced by creating a flexible object-action order such that the user could utter or select a command with a pen followed by the object which was to be enacted upon, or the other way round (e.g., New Message or Message New). The flexible action-object interaction design combined with voice and pen input led to eight possible action-object-modality combinations. The complexity of the multimodal interface was further reduced by making generic commands such as New applicable across corresponding objects. Use of generic commands led to a simplification of menu structures by reducing the number of instances in which actions appeared. In this manner, more content information could be made visible and consistently accessible via pen and speech input. Results of a controlled experiment indicated that the shortest task completion times for the eight possible input conditions were when speech-only was used to refer to an object followed by the action to be performed. Speech-only input with action-object order was also relatively fast. In the case of pen input-only, the shortest task completion times were found when an object was selected first followed by the action to be performed. In multimodal trials in which both pen and speech were used, no significant effect was found for object-action order, suggesting benefits of providing users with a flexible action-object interaction style in multimodal or speech-only systems.     

A Unified, Object-Oriented Graphics System and Software Architecture for Visualising CAD/CAM Presentations

Most object-oriented graphics systems (OOGS) either support general purpose graphics capabilities (drawing editors, ray tracing, etc.), or have targeted specific areas of computer graphics, for example, interface building, animation, and visualisation. However, the key concept in CAD/CAM graphics systems is that of presentations—pictures that convey information about products or parts of products. CAD/CAM graphics systems that create and manipulate presentations have specific graphical requirements that have not been addressed by existing OOGS. These requirements include the support of CAD/CAM concepts such as drawings, views and view-specific graphics, and layers, and the ability to allow external geometric modelers to be linked into the presentation. We describe an object-oriented presentation architecture that allows the application to describe the structure of the CAD/CAM presentation. This architecture provides a unified interface to drawings, views, layers and entities, and allows linkage to models created with an external geometric modeler, while isolating applications from details of the underlying traditional graphics rendering systems. It is our hope that this architecture provide the foundation for the architecture of next generation OOGS. We also present an implementation of this architecture and compare it to other OOGS.     

嵌入式图形用户界面系统的设计与实现

针对GUI开发方式存在规范化和标准化程度低,代码复用性差等问题,提出了一种轻量级嵌入式图形用户界面LIGUI (lightweight embedded GUI)系统.系统采用分层结构和模块化设计思想,通过输入输出抽象层屏蔽底层设备的不一致,采用双向链表组织窗口结构,在应用接口层提供配置选项,支持系统裁剪和扩展.实际应用表明,该系统占用资源少,适用于低端配置的应用环境、系统可移植、易裁剪和配置,可以满足嵌入式环境应用需求.