Integration Model of Eye-Gaze, Voice and Manual Response in Multimodal User Interface

This paper reports the utility of eye-gaze,voice and manual response in the design of multimodal user interface.A device-and application-independent user interface model(VisualMan)of 3D object selection and manipulation was developed and validated in a prototype interface based on a 3D cube manipulation task.The multimodal inpus are integrated in the prototype interface based on the priority of modalities and interaction context.The implications of the model for virtual reality interface are discussed and a virtual environment using the multimodal user interface model is proposed.     

A Domain Knowledge Driven Approach for User Interface Software Development

A domain knowledge driven user interface development approach is described.As a conceptual design of the user interface,the domain knowledge defines the user interface in terms of objects,actions and their relationships that the user would use to interact with the application system.It also serves as input to a user interface management system (UIMS) and is the kernel of the target user interface.The principal ideas and the implementation techniques of the approach is discussed.The user interface model,user interface designer oriented high-level specification notatiopn,and the transformation algorithms on domain knowledge are presented.     

Recent advances on dynamic learning from adaptive NN control

Notice that intelligent control is developed based on the learning and control abilities of human beings. As such, an ideal intelligent control system should not only have the ability to adapt to complex dynamic environments, but also the ability to imitate human learning. The learning ability means to acquire the knowledge of uncertain closed-loop dynamics from dynamic environments, and use the stored experience knowledge to improve control performance [1,2]. Such an autonomous learning has been viewed as one of the most important and indispensable capabilities to achieve a high degree of system autonomy. The existing intelligent control schemes, including adaptive neural control and adaptive fuzzy control, are usually exclusively on the system stability by the use of online adjustment parameters to estimate the unknown dynamics, in which the exponential convergence of the estimated parameters is not a major concern. As a result, most of existing intelligent control schemes still need to readjust the controller parameters for the same or similar control tasks. In this sense, as shown in \cite{3}, the learning ability of intelligent control is actually very limited and still needs the further research. However, it is very challenging to acquire and express the knowledge of unknown dynamics in a nonstationary or dynamic environment....     

An Environment for Rapid Prototyping of Interactive Systems

This paper shows an environment which supports the development of multi-thread dialogue interactive systems.The environment includes several tools and run-time support programs for the design and implementation of the user interface of an interactive system.First,methods of user interface specification with Elementary Nets are discussed.Then,the syntax of a user interface specification language based on Elementary Nets and the pre-compiler for the language as well as a graphic editor for Elementary Nets construction are described.Finally,an example is given to illustrate the design process of a user interface.     

Use Case for the Development of a Mobile Learning Application

This paper presents use cases developed for a mobile learning application named MLEA （mobile learning environment adapter）, developed in collaboration with the Universidad do Vale do Rio dos Sinos-Unisinos, Brazil and UTP （Universidad Tecnologica de Panama）. MLEA allows users to access, through a single platform, courses in a content management system--Moodle. The objective of this project is to create a communication platform that allows users of Android based mobile devices to use services of the virtual campus of UTP. The work team is composed of six professors and researchers in Panama and three in Brazil, with voluntary students in both countries. A complete, step by step use case is documented, including requirement specifications, interface designs based on Android patterns, as well as programming patterns used for the communication between mobile devices and Moodle platform.     

Human knowledge acquisition from 3D interaction in virtual environments

Recently,virtual environment (VE) based design and planning,which is a kind of interaction intensive computing,has shown great potential.Most users of such systems are specialists or technicians and their 3D interactions with the system embed a great deal of knowledge and skills.Thus,how to integrate human knowledge and skills into VE is a great challenge to researchers in the human computer interaction field.This paper proposes a method for acquiring user knowledge from VE.The main ideas and work include,abstracting the interactive process and formalizing the interactive semantics,as well as providing a semantic model of an interactive information repository,from which user interactive processes can be retrieved and all kinds of application logics can be established.A virtual assembly planning system is introduced as an example of a practical application of this method.Experiments show that the related models can well capture user knowledge and retrieve the interaction process.     

Construction of a new adaptive wavelet network and its learning algorithm

A new adaptive learning algorithm for constructing and training wavelet networks is proposed based on the time-frequency localization properties of wavelet frames and the adaptive projection algorithm. The exponential convergence of the adaptive projection algorithm in finite-dimensional Hilbert spaces is constructively proved, with exponential decay ratios given with high accuracy. The learning algorithm can sufficiently utilize the time-frequency information contained in the training data, iteratively determines the number of the hidden layer nodes and the weights of wavelet networks, and solves the problem of structure optimization of wavelet networks. The algorithm is simple and efficient, as illustrated by examples of signal representation and denoising.     

Software Engineering in the Application of Virtual Instrument-Based Dynamics Testing System for UAV

Complex dynamics testing system for UAV can be constructed based on the virtual instrument and the software is the core LabVIEW is a graphical-based programming language with a convenient user interaction and shorter development cycle than the text-based programming language. This paper uses LabVIEW as the software development platform of the dynamics testing system for UAV. The idea of software engineering is used in the design of the testing system for UAV, and then is used to improve the reliability, stability and scalability of the testing system. The experimental results show that this system is practical, convenient to operate, able to meet the needs of the engineering and teaching for UAV research and development.     

StoryMap: An Efficient Sketch-Based Interface for Video Navigation

With the exploration of video data, it is difficult for people to navigate freely and make full use of amount of video data efficiently in order to get useful information or knowledge with an aim to fulfill special tasks such as visual analysis. In this paper, we propose a novel sketch based interface which is called StroyMap for video content representation and navigation under the guideline of distributed cognition. Instead of emphasizing on details, the sketches visualized the essential content of video effectively. In order to meet different requirement for video exploration tasks, StoryMap provides users a variety of navigation ways, such as path redirection, map tagging, zoom in and zoom out. During the whole interaction process, the system will collect user''s operation list, construct user model as a guide to understand interaction between people and computer. Experimental evaluation results show that StoryMap plays more effiective role in conveying and navigating video content compared with existing methods.     

User Modeling for Adaptive News Access

We present a framework for adaptive news access, based on machine learning techniques specifically designed for this task. First, we focus on the system's general functionality and system architecture. We then describe the interface and design of two deployed news agents that are part of the described architecture. While the first agent provides personalized news through a web-based interface, the second system is geared towards wireless information devices such as PDAs (personal digital assistants) and cell phones. Based on implicit and explicit user feedback, our agents use a machine learning algorithm to induce individual user models. Motivated by general shortcomings of other user modeling systems for Information Retrieval applications, as well as the specific requirements of news classification, we propose the induction of hybrid user models that consist of separate models for short-term and long-term interests. Furthermore, we illustrate how the described algorithm can be used to address an important issue that has thus far received little attention in the Information Retrieval community: a user's information need changes as a direct result of interaction with information. We empirically evaluate the system's performance based on data collected from regular system users. The goal of the evaluation is not only to understand the performance contributions of the algorithm's individual components, but also to assess the overall utility of the proposed user modeling techniques from a user perspective. Our results provide empirical evidence for the utility of the hybrid user model, and suggest that effective personalization can be achieved without requiring any extra effort from the user.     

A graph-based framework for multiparadigmatic visual access todatabases

Describes an approach for multiparadigmatic visual access integration of different interaction paradigms. The user is provided with an adaptive interface augmented by a user model, supporting different visual representations of both data and queries. The visual representations are characterized on the basis of the chosen visual formalisms, namely forms, diagrams and icons. To access different databases, a unified data model called the “graph model” is used as a common underlying formalism to which databases, expressed in the most popular data models, can be mapped. Graph model databases are queried through the adaptive interface. The semantics of the query operations is formally defined in terms of graphical primitives. Such a formal approach permits us to define the concept of an “atomic query”, which is the minimal portion of a query that can be transferred from one interaction paradigm to another and processed by the system. Since certain interaction modalities and visual representations are more suitable for certain user classes, the system can suggest to the user the most appropriate interaction modality as well as the visual representation, according to the user model. Some results on user model construction are presented     

Using Distributable User Interfaces in CSCL In-Situ Classrooms

Distributable user interfaces enable users to distribute user interface interaction objects (i.e. panels, buttons, input fields, checkboxes, etc.) across different displays using a set of distribution primitives to manipulate them in real time. This work presents how this kind of user interfaces facilitates the computer supported collaborative learning in modern classrooms. These classrooms provide teachers and students with display ecosystems consisting of stationary displays, such as smart projectors and smart TVs as well as mobile displays owned by teachers and students, such as smartphones, tablets, and laptops. The distribution of user interface interaction objects enables teachers to modify the user interface interaction objects that are available to students in real time to control and promote the collaboration and participation among them during learning activities. We propose the development of this type of applications using an extension of the CAMELEON reference framework that supports the definition of UI distribution models. The Essay exercise is presented as a case of study where teachers control the collaboration among students by distributing user interface interaction objects.     

Teaching the User By Learning From the User:Personalizing Movement Control in Physical Human-robot Interaction

This paper proposes a novel approach for physical human-robot interactions (pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior of each user in coping with different tasks, where lower performance results in higher intervention from the robot. This personalized physical human-robot interaction (p2HRI) method incorporates adaptive modeling of the interaction between the human and the robot as well as learning from demonstration (LfD) techniques to adapt to the users' performance. This approach is based on model predictive control where the system optimizes the rendered forces by predicting the performance of the user. Moreover, continuous learning of the user behavior is added so that the models and personalized considerations are updated based on the change of user performance over time. Applying this framework to a field such as haptic guidance for skill improvement, allows a more personalized learning experience where the interaction between the robot as the intelligent tutor and the student as the user, is better adjusted based on the skill level of the individual and their gradual improvement. The results suggest that the precision of the model of the interaction is improved using this proposed method, and the addition of the considered personalized factors to a more adaptive strategy for rendering of guidance forces.      

From user interface design to the support of intelligent and adaptive interfaces: an overhaul of user interface software infrastracture

Research on adaptive interfaces in the past has lacked support from user interface tools which allow interfaces to be easily created and modified. Also, current user interface tools provide no support for user models which can collect task-oriented information about users. Developing an adaptive interface requires a user model and an adaptation strategy. It also, however, requires a user interface which can be adapted. The latter task is often time-consuming, especially in relation to more sophisticated user interfaces.

The paper presents a user interface design environment, UIDE, which has a different software infrastracture. Designers use high-level specifications to create a model of an application and links from the application to various interface components. The model is the heart of all the design and run-time support in UIDE, including automatic dialog sequencing and help generation. UIDE provides automatic support for collecting task-oriented information about users, by the use of its high-level specifications in its application model as a basic construct for a user model. Some examples of adaptive interfaces and adaptive help are presented that use the information that is collectable in UIDE.     

Design, implementation and evaluation of an adaptive user interface

Software systems developed in recent years are becoming increasingly powerful, but in most cases they tend to abandon the user to deal with the complexity of the system alone. There is an immense need for systems with individual, context-sensitive support. The paper discusses the development of such a support system with special focus on the user interface. The extension of an existing and complex program. Excel, to an adaptive and customizable system is described. This extension is called Flexcel, and it has been implemented and evaluated.

The primary aim of this extension is to provide a convenient environment for adapting Excel's user interface to particular users and their current tasks. For example, it allows the user to define new menu for actions that are frequently needed and normally require a tedious dialog step. Flexcel also analyzes the user's interaction style and presents adaptation suggestions. A critique component supports the user in his/her problem solving and learning activities related to the adaptability of Flexcel's user interface.     

动态调度系统中自适应用户界面的设计与实现

该文以炼化企业的动态调度系统为应用背景，以面向动态调度的自适应用户界面的设计与实现为主要研究内容，分析了当前炼化企业动态调度系统用户操作的流程以及界面的需求并建立起相应的数学模型。借鉴自适应用户界面的相关研究成果，设计了自适应用户界面的体系结构以及用户的动态行为过程，针对流程行业的应用特点提出了利用产生式推理和基于上下文的用户意图捕捉的方法，最后给出应用实例。文中所述方法已经成功在企业中得到应用。使用结果表明，该方法能够有效地开发动态调度系统的自适应用户界面，获得满意效果。     

Avoiding misconstruals in database systems: a default logicapproach

A cooperative interface that, using suitable user models, alters the processing of the user's queries to include additional information that will block faulty inferences is described. In a sense, the interface actively teaches the user facts about the database not explicitly asked for. User interaction with the database then becomes a learning and discovery process guided by the queries posed to the interface. A semantics for user models is introduced that captures, with the help of default logic, the nonmonotonic behavior users normally exhibit. Results showing that the cooperative interface generates enough additional information to block all faulty inferences are presented     

Machine learning: A tool to support usability?

One problem facing designers of interactive systems is catering to the wide range of users who will use a particular application. Understanding the user is critical to designing a usable interface. There are a number of ways of addressing this problem, including improved design methodologies using ''intuitive'' interface styles, adaptive interfaces, and better training and user support materials. In this article, we argue that each of these solutions involves pattern recognition in one form or another and that machine learning can therefore aid designers of interactive systems in these areas. We report on experiments that demonstrate the potential of machine learning to user modeling that has application to two of these areas in particular: adaptive systems and design methodologies.     

Adaptive training interfaces for less-experienced,elderly users of electronic devices

A great number of complex electronic devices are now part of our everyday lives. While many of us learn to handle these products by trial and error; others, especially older users with little experience in using electronic devices, need support. In order to allow the user maximum flexibility in terms of learning time and location, a training programme is presented which is implemented as part of the software embedded in the product itself. Particular focus is placed on the effect of adaptive training on learning. In this study, the training versions differed in their ability to adjust their complexity to the user's experience (adaptive user interface complexity) and their capability to support the learner by prompting them during the learning process (adaptive training advice). The results show that the adjustment of complexity had a positive effect on users’ experience: elderly users who trained with an adaptive interface were more successful in learning to use a mobile phone. Adaptive training advice, however, was found to have no significant effects on learners’ success and reduced their self-efficacy. This work offers guidelines on how to design integrated training applications for electronic devices that successfully help elderly users with little prior experience.