Haptic User Interfaces for the Visually Impaired: Implications for Haptically Enhanced Science Learning Systems

The overall quality of haptic user interfaces designed to support visually impaired students' science learning through sensorial feedback was systematically studied to investigate task performance and user behavior. Fourteen 6th- to 11th-grade students with visual impairments recruited from a state-funded blind school were asked to perform three main tasks (i.e., menu selection, structure exploration, and force recognition) using haptic user interfaces and a haptic device. This study used several dependent measures that are categorized into three types of variables: (a) task performance including success rate, workload, and task completion time; (b) user behavior defined as cursor movements proportionately represented from the user's cursor positional data; and (c) user preference. Results showed that interface type has significant effects on task performance, user behavior, and user preference, with varying degree of impact to participants with severe visual impairments performing the tasks. The results of this study as well as a set of refined design guidelines and principles should provide insights to the future research of haptic user interfaces that can be used when developing haptically enhanced science learning systems for the visually impaired.     

Plan recognition for interface agents

Interface agents are computer programs that provide personalized assistance to a user dealing with computer based applications. By understanding the tasks the user performs in a software application an interface agent could be aware of the context that represents the user’s focus of attention at each particular moment. With this purpose, plan recognition aims at identifying the plans or goals of a user from the tasks he (for simplicity, we use “he” to refer to the user, but we do not mean any distinctions about sexes) performs. A prerequisite for the recognition of plans is knowledge of a user’s possible tasks and the combination of these tasks in complex task sequences, which describes typical user behavior. Plan recognition will enable an interface agent to reason about what the user might do next so that it can determine how to assist him. In this work we present the state of the art in Plan Recognition, paying special attention to the features that make it useful to interface agents. These features include the ability to deal with uncertainty, multiple plans, multiple interleaved goals, overloaded tasks, noisy tasks, interruptions and the capability to adapt to a particular user.     

Permissive user interfaces

User interfaces often only support one way to do a task when the physical interface or the requirements of the task would permit other ways. In contrast, a user interface that supports multiple approaches is permissive. This paper argues that permissive user interfaces are easier to use—and even when they are not applicable for particular applications, considering permissiveness is a productive design heuristic.Many user interfaces are difficult to use yet very easily demonstrated or explained by experts—with the results that users become frustrated because hindsight makes usability problems look like the user's own fault. The lack of permissiveness in such user interfaces explains this paradox.     

Beyond performance: Feature awareness in personalized interfaces

Personalized graphical user interfaces have the potential to reduce visual complexity and improve interaction efficiency by tailoring elements such as menus and toolbars to better suit an individual user's needs. When an interface is personalized to make useful features more accessible for a user's current task, however, there may be a negative impact on the user's awareness of the full set of available features, making future tasks more difficult. To assess this tradeoff we introduce awareness as an evaluation metric to be used in conjunction with performance. We then discuss three studies we have conducted, which show that personalized interfaces tradeoff awareness of unused features for performance gains on core tasks. The first two studies, previously published and presented only in summary, demonstrate this tradeoff by measuring awareness using a recognition test of unused features in the interface. The studies also evaluated two different types of personalized interfaces: a layered interfaces approach and an adaptive split menu approach. The third study, presented in full, focuses on adaptive split menus and extends results from the first two studies to show that different levels of awareness also correspond to an impact on performance when users are asked to complete new tasks. Based on all three studies and a survey of related work, we outline a design space of personalized interfaces and present several factors that could affect the tradeoff between core task performance and awareness. Finally, we provide a set of design implications that should be considered for personalized interfaces.     

Object-oriented specification of user interfaces

This paper presents an object-oriented approach for the specification of graphical user interfaces. Specification starts with the analysis of the end user's operations. The user interface is then designed on the basis of this analysis. Operation analysis is followed by structure and component specification which presents the dialogue structure of the application and the contents of each dialogue. Visualization produces the final screen layouts, and task specification documents the usage of the user interface for the purpose of creating user's guides. The method presented in this paper makes it easier for a designer to take the end user's needs into account. Still, it does not automatically guarantee good quality user interfaces. The top-down nature of the method allows the designer to concentrate on the most important aspects of the user interface and split the design procedure into manageable pieces. Also, the visibility of the process allows the designer to communicate with other people while specifying the user interface. This paper connects the method with the object-oriented specification of entire applications. It briefly explains the connections with object-oriented analysis and design, and demonstrates how to implement the specified user interface in an object oriented fashion. The approach presented in this paper is being applied in the development of a large network management system with about two million lines of C++ code running in the XII environment. Still, the method does not require the specification being implemented with any specific windowing system. The only requirement is that the user interface is based on graphical elements, such as dialogues, push-buttons and text fields.     

Evaluation of a Geospatial Annotation Tool for Unmanned Vehicle Specialist Interface

Map-based interfaces have been developed to support collaborative control of unmanned vehicles (i.e., robots). Annotation on the map (or geospatial annotation) has been proposed as an effective way to support team collaboration; however, there is a lack of research focused on the design of geospatial annotation tools to promote usability and task performance. The utility of location reference in geospatial annotations for communication and information sharing is the focus of this article. Two annotation tool designs were developed. The annotation contents were directly anchored on the map in the first design, whereas in the second design annotations were summarized in a separate panel on the interface. Evaluation participants followed instructions from a simulated team leader and assigned unmanned vehicles to different tasks for two simulated scenarios that include searching for victims and collecting hazardous materials samples. The results demonstrate the potential of using geospatial annotations to enrich communication and support map-based unmanned vehicle control. Participants appreciated the direct location reference feature of the first design and had generally shorter response time, but felt that the second design provided better usability and lower task workload. These results suggest that the user experience depends on the manner of obtaining information from the annotation tools, and the integration of the tool with user's task flow and other interface components, such as the map display. The presented results can be used as a basis for designing geospatial annotation tools for team collaboration that better fit user needs and preferences.     

FROM PLAN CRITIQUING TO CLARIFICATION DIALOGUE FOR COOPERATIVE RESPONSE GENERATION*

Recognizing the plan underlying a query aids in the generation of an appropriate response. In this paper, we address the problem of how to generate cooperative responses when the user's plan is ambiguous. We show that it is not always necessary to resolve the ambiguity, and provide a procedure that estimates whether the ambiguity matters to the task of formulating a response. The procedure makes use of the critiquing of possible plans and identifies plans with the same fault. We illustrate the process of critiquing with examples. If the ambiguity does matter, we propose to resolve the ambiguity by entering into a clarification dialogue with the user and provide a procedure that performs this task. Together, these procedures allow a question-answering system to take advantage of the interactive and collaborative nature of dialogue in order to recognize plans and resolve ambiguity. This work therefore presents a view of generation in advice-giving contexts which is different from the straightforward model of a passive selection of responses to questions asked by users. We also report on a trial implementation in a course-advising domain, which provides insights on the practicality of the procedures and directions for future research.     

Can eye-tracking data be measured to assess product design?: Visual attention mechanism should be considered

Eye tracking probes user's perception of real-time reaction to products, while conventional methods (i.e. interviews, focus group, questionnaires and so on) have generally failed because they depend on users' willingness and competency to describe how they feel when they are exposed to a product. Two tasks were designed to explore the indexes of eye movement that can reflect user experience of product, and analyse the attention captured by product attributes and goal-oriented. In task one, participants just browsed two smart phone pictures and evaluated the whole user experience. Binary choices were used in task two to ask participants to select the smart phone picture with higher user experience and then click the mouse. The results showed that in the browsing task, participants had shorter time to first fixation for the smart phone picture with higher level of user experience than the lower. And pupil dilated significantly when participants browse smart phone picture with lower level of user experience. In goal-oriented task, participants' attentions were dominated by visual perception of task driven, mainly reflected on longer fixation time and larger pupil diameter when looking at the smart phone with higher level of user experience. These results support the notion that we cannot assess product design just by several eye-movement indexes without considering the effects of visual attention mechanism.Relevance to industryThe appearance of product plays an important role to attract user's attention and stimulate their intention to experience. And vision is the main channel for users to obtain product information. Hence a thorough research on the inherent mechanism of vision perception can provide technical support for product designers, which in turn can attract more consumers to experience the product, even buy it. Moreover, the seller can find out the real buyers and predict their desired products by tracking user's eyes.     

Applying Plan Recognition Algorithms To Program Understanding

Program understanding is often viewed as the task of extracting plans and design goals from program source. As such, it is natural to try to apply standard AI plan recognition techniques to the program understanding problem. Yet program understanding researchers have quietly, but consistently, avoided the use of these plan recognition algorithms. This paper shows that treating program understanding as plan recognition is too simplistic and that traditional AI search algorithms for plan recognition are not suitable, as is, for program understanding. In particular, we show (1) that the program understanding task differs significantly from the typical general plan recognition task along several key dimensions, (2) that the program understanding task has particular properties that make it particularly amenable to constraint satisfaction techniques, and (3) that augmenting AI plan recognition algorithms with these techniques can lead to effective solutions for the program understanding problem.     

Using dynamic user models in the recognition of the plans of the user

This paper is concerned with information-seeking dialogues in a restricted domain (we consider a consultation system for a Computer Science Department, delivering information about the various tasks that the users may want to perform: for example, how to access the library, get information about the courses of the Department, etc.) and presents a framework where a plan recognition and a user modeling component are integrated to cooperate in the task of identifying the user's plans and goals. The focus of the paper is centered on the techniques used for building the user model and exploiting it in the determination of the user's intentions. For this task, we use stereotypes and we propose some inference rules for expanding the user model by inferring the user's beliefs from both the sentences s/he utters and the information stored in the plan library of the system, that describes the actions in the domain. Moreover, we introduce some disambiguation rules that are applied to the information in the user model for restricting the set of ambiguous hypotheses on the user's plans and goals to the most plausible ones. This also simplifies a further clarification dialogue if it is necessary for a precise identification of the user's intentions.     

Planning for tourism routes using social networks

Traveling recommendation systems have become very popular applications for organizing and planning tourist trips. Among other challenges, these applications are faced with the task of maintaining updated information about popular tourist destinations, as well as providing useful tourist guides that meet the users preferences. In this work we present the PlanTour, a system that creates personalized tourist plans using the human-generated information gathered from the minube¹ traveling social network. The system follows an automated planning approach to generate a multiple-day plan with the most relevant points of interest of the city/region being visited. Particularly, the system collects information of users and points of interest from minube, groups these points with clustering techniques to split the problem into per-day sub-problems. Then, it uses an off-the-shelf domain-independent automated planner that finds good quality tourist plans. Unlike other tourist recommender systems, the PlanTour planner is able to organize relevant points of interest taking into account user’s expected drives, and user scores from a real social network. The paper also highlights how to use human provided recommendations to guide the search for solutions of combinatorial tasks. The resulting intelligent system opens new possibilities of combining human-generated knowledge with efficient automated techniques when solving hard computational tasks. From an engineering perspective we advocate for the use of declarative representations of problem solving tasks that have been shown to improve modeling and maintenance of intelligent systems.     

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems

Evolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness.     

The intelligent help system COMFOHELP

The paper is concerned with the question of whether and under what conditions active help systems with plan recognition components that have been developed in the environment of artificial intelligence research are able to prove their value in the real context of commercial application programs. The question is investigated using the development of the COMFOHELP intelligent help system as an example. COMFOHELP supports the COMFOTEX graphical text processing program and has been developed by the Linguistic Information Science Group at the University of Regensburg since 1988. The system recognizes erroneous and suboptimal plans pursued by the user by analyzing the dialog history and comparing them with the correct plan for achieving the user's goal.Section 2 discusses the research situation and elaborates on those problems which up to now prevented research concepts for plan recognition and intelligent help systems from being practically applied. Testing error situations empirically is a first prerequisite since potential erroneous plans can only be established in real-world tests. The second prerequisite is a special system architecture which counteracts the problem of ambiguities in plan recognition. Section 3 introduces a first still restricted prototype version of COMFOHELP whose efficiency was verified in a statistical hypothesis test. The users performing their text processing tasks with the support of COMFOHELP came off significantly better than members of a reference group working without the intelligent help. Section 4 shows that the proposed COMFOHELP system architecture is reconfirmed by the results of extensive empirical investigations (with more than 100 users) of erroneous plans when using a more complex version of COMFOTEX. The architecture still proves to be worthwhile even when functionality is increased by a factor of three to four.     

Modeling the user's conceptual knowledge in BGP-MS,a user modeling shell system1

The paper describes the modeling of a user's conceptual knowledge in the general user modeling shell system BGP-MS. On the one hand, BGP-MS is a workbench for the develment of a user model in a particular application domain. It supports the definition of the architecture of the individual user model, and of the architecture and the contents of user stereotypes. A rich representation language for conceptual knowledge, a partition mechanism, and flexible graphics-based interfaces are at the disposal of the user model developer. On the other hand, BGP-MS is a runtime user modeling tool aimed at fulfilling central tasks of a user modeling component in an application system. In this mode, the system offers a functional interface for accessing and updating the model of the current user, as well as basic domain-independent inference mechanisms, support for inferences defined by the developer, and a customizable stereotype management utility. The usefulness of BGP-MS will be demonstrated by illustrating how it can support the user modeling of various recently developed application systems.     

面向肌电信号的虚拟现实提线木偶动画研究

泉州提线木偶属于首批中国非物质文化遗产，是中华传统文化的实现形式之一.然而，由于木偶体积庞大，携带与操作不便，直接限制了其受众群体.为了实现提线木偶的有效传承与保护，设计了基于手势识别的虚拟现实提线木偶动画方案，构建了基于MYO臂环肌电信号的人体生理信号控制动画原型系统，应用两个用户实验验证了算法的高精确性与易操控性.首先，通过低通滤波与平滑实现多通道肌电信号数据的信号处理.其次，提取八通道时域特征与时频域特征，并通过线性判别器将其降维为六维特征向量，实现特征间关联性消除与算法鲁棒性增强.最后，构造多分类支撑向量机实现特征向量，确定手势识别结果.实验验证算法离线动作平均识别准确率为95.59%，实时动作平均识别准确率达到90.75%，在1.1s左右完成手势识别.面向提线木偶任务，构造了两个用户体验任务，普通用户人群中，木偶动作识别率较高，用户使用意愿、易学性等方面，系统性能亦显著高于真实木偶操控；专业用户在承认系统可用性的同时，具有较高的接受度.用户任务表明该设计满足了手势识别实时性和准确性的要求，具有良好的交互性和趣味性.相关研究可以广泛地应用于计算机动画等类似的系统，对于体验和保护提线木偶具有现实意义.     

面向大众智能化管线统计功能设计与实现

在地下管线系统建设项目中,为了提高地下管线管理系统的适用性和满足该领域非专业用户的实际管理需求,针对传统管线统计功能对使用人员专业要求门槛高和用户体验感差等方面的不足,设计和实现了一种面向大众智能化管线统计功能及方法。用户充分依据实际业务需求在界面上进行简便操作,系统后台会智能化地识别用户的操作步骤,经过内嵌的模块解析用户的操作意图,再将其合理组合,编译成数据库引擎能够识别的查询统计命令,以获得用户所需要的统计结果并予以展示。系统功能界面提供了通俗易懂的操作模式,以所见即所得的方式,全程操作无须用户编写SQL查询语句即可进行自由组合的统计,系统设计的内嵌智能化识别模块消除了人机交互之间的语言障碍,解决了用户对管线统计功能的低门槛操作、高自由度统计的需求,系统功能的简便操作进一步提升了管线系统中人机交互的体验感,为管线系统的推广提供技术支持。     

Emotion recognition from physiological signals using wireless sensors for presence technologies

In this article we describe a new approach to enhance presence technologies. First, we discuss the strong relationship between cognitive processes and emotions and how human physiology is uniquely affected when experiencing each emotion. Secondly, we introduce our prototype multimodal affective user interface. In the remainder of the paper we describe the emotion elicitation experiment we designed and conducted and the algorithms we implemented to analyse the physiological signals associated with emotions. These algorithms can then be used to recognise the affective states of users from physiological data collected via non-invasive technologies. The affective intelligent user interfaces we plan to create will adapt to user affect dynamically in the current context, thus providing enhanced social presence.     

A Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface

In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user's hand. This paper describes the use of a hand input device for interacting with a 3D graphical application. A dynamic gesture language, which allows users to teach some hand gestures, is presented. Furthermore, a user interface integrating the recognition of these gestures and providing feedback for them, is introduced. Particular attention has been spent on implementing a tool for easy specification of dynamic gestures, and on strategies for providing graphical feedback to users' interactions. To demonstrate that the introduced 3D user interface features, and the way the system presents graphical feedback, are not restricted to a hand input device, a force input device has also been integrated into the user interface.