基于惯性感应的手机交互模型研究及应用分析

传感器技术在人机交互领域催生了新型的3D输入方式,并促使不可见交互方式广泛应用.在分析现有交互模型的基础上,提出了基于惯性感应的手机交互模型,具有层次分明,扩展性好等特点.在此基础上提出一种手机交互系统概念模型并分析了基于惯性感应的手机应用.     

盲人触觉图形显示器的交互体验研究

在互联网时代,盲人由于视力缺失,通过触听觉对图形图像的认知需求越来越强烈.文中通过对触觉和现有触觉图形显示器的梳理分析,围绕其交互技术和用户体验进行研究.首先从人的触觉生理角度论证了触觉图形显示器的可用性;然后列举了主要的触觉图形显示器的研发原型或产品,梳理了各种交互技术及参数的对比,并以用户体验的角度,通过实验分析了影响体验的重要设计因素.最后针对大数据时代,以盲人对产品的体验为中心设计了一套盲人触觉显示器的交互原则,为触觉图形显示器的设计提供了参考.     

基于上下文的多通道语义融合

多通道交互已经成为人机交互领域一个重要的研究方向,对多通道输入的信息进行语义融合是其中的关键问题.提出了一种将上下文与传统语义融合相结合的方法,并将这一方法应用在面向儿童使用的讲故事原型系统中.儿童通过受限的自然语言、笔的勾画等方式进行故事创作.结果表明,笔和语音结合的多通道交互方式能够使得儿童更方便、自然地与计算机进行交互,结合上下文的语义融合方法能够在一定程度上修正输入错误.     

基于智能手机的三维交互技术研究

利用智能手机的多通道输入和输出功能可为解决大屏幕远距离交互提供了一条有效途径.通过分析大屏幕远距离交互的特点,给出了一个基于智能手机的三维交互框架;针对利用惯性传感器的计算智能手机空间姿态问题,结合交互的特点提出了一种实时、快速、准确地解算手机的空间姿态的算法;基于手机的空间姿态和触摸交互,针对大屏幕交互中二维交互与三维交互分别设计了2类交互技术,并给出3个应用实例.最后通过实验对这3个实例进行了实验评估,结果表明智能手机在大屏幕、远距离交互中是合理与有效.     

自然交互研究——笔式简谱编辑器的设计与实现

自然交互是指用户可以用自己熟悉的日常技能与计算机进行交互。中国有大量的音乐爱好者使用简谱,但是针对简谱编辑的编辑软件却相对较少,并且现有软件的交互方式不够自然,降低了用户对软件的使用效率。他们需要一种更自然、更方便使用的简谱编辑软件。该文介绍了一个基于笔式输入的简谱编辑器原型系统的设计和实现。基于对现有软件、简谱特性和可利用的新技术的综合分析,设计并实现了这个原型系统。对此原型系统进行了非正式的用户实验之后,对结果进行了分析,并提出了修改的方案。     

基于XML的结构化电子病历系统设计

阐述了两种结构化数据采集的方法:自然语言处理(NLP)和结构化数据输入(SDE).基于上述两种方法,设计了一个结构化电子病历系统(EPR)的原型,用XML技术来描述和实现知识库,使用Microsoft.NET的XML数据流技术实现病历数据的输入、存储和展现,并使用XSL技术实现Web方式的病历浏览.设计表明,利用XML技术解决病历的自由输入并和结构化输入相结合,是可行的技术方案;它为电子病历的研究提供了新的途径.     

面向多用户交互桌面的界面管理系统

针对多用户交互桌面的界面管理问题,通过需求分析、界面建模、原型实现与用户研究四个阶段的研究,提出了一个完整的界面管理方案.首先基于对交互式桌面水平显示、直接输入以及多用户使用三个特性的分析,提出了桌面界面管理中输入管理、布局管理、权限管理和冲突管理四个方面的新需求,这些显著区别于台式计算机上的界面管理.在此基础上,设计了层次式的桌面交互界面模型,它自底向上由处理多用户输入的输入管理层、控制交互界面的对象管理层、以及基于树状结构组织的图形对象层组成.进而,实现了多用户桌面的交互界面管理系统uTablePlatform,并开展了实际的用户研究.4 组测试者在uTablePlatform 上分别完成了一次旅行规划的任务,观察和问卷结果表明本文设计并实现的界面管理系统可以有效的帮助用户完成桌面交互任务.     

基于智能手机传感器与触摸屏的手势交互技术

由于以图像为手势信息载体的交互方式易受环境影响,设计实现了一种基于智能手机触摸屏与内置传感器的手势交互方案。首先通过手机内置MEMS传感器及触摸屏获取手势信息;然后分析手势的运动学特征,实时提取手势的信息(角速度、加速度、触摸屏数据),利用多变量决策树分类器完成对手势分类与识别;最后给出了一个原型系统。实验结果表明,该方案满足实时人机交互需求。     

面向指-笔混合输入的交互原语和交互模型研究

随着触摸设备的普及, 指、笔输入已成为移动办公的一种主流输入方式. 然而现有的应用主要以手指或笔通道中的一种作为输入, 限制了用户的交互空间. 同时, 现有的指-笔混合研究主要关注串行的指-笔分工协作和特定交互任务的并行处理, 并没有系统地考虑并行协作机制和不同通道输入之间的意图关联. 为此, 提出一种面向指-笔混合输入的交互模型; 结合用户指-笔协作的行为习惯定义指-笔混合交互原语, 拓展指、笔交互空间; 继而利用部分可观察马尔可夫决策过程提出基于时序信息的指-笔混合输入意图提取方法, 用以增量式地提取多义性交互原语的交互意图. 最后通过一个用户实验说明指-笔混合输入的优势.     

任务感知双原型网络的人物交互少样本识别

人物交互(HOI)识别是计算机视觉领域的重要研究热点.随着深度学习在图像分类任务中的巨大成功,人物交互识别任务也取得重大进展,但样本不平衡和组合爆炸问题仍是制约当前人物交互识别任务性能的关键挑战.由此,将人物交互识别任务与少样本学习相结合,将人物交互识别任务定义为一个少样本任务,并提出了任务感知双原型网络(TDP-Net)来解决少样本人物交互任务.具体地,首先使用图方法为每个任务生成语义感知的任务表示作为任务的先验信息,并使用语义图注意力模块(SGA-Module)生成注意力权重,对特征图中不同区域进行不同重要程度的关注,以适应不同任务条件下的映射关系,实现在新任务中自动推理.此外,还设计了一个双路原型模块(DP-Module)以分别产生交互类别的动作类原型和物体类原型,并分别对动词和名词进行分类.通过分别为动作和物体建立类原型,有效地分离了动作和物体间复杂的视觉关系.同时由于人物交互类别之间具有相似性,可通过重新组合动作和物体类别将知识迁移到新的交互类别中.实验结果表明,该模型在人物交互少样本任务上的平均准确率比基线方法在两个实验设置上分别提高了3.2个百分点和15.7个百分点,验证了TDP-Net在少样本人物交互任务中的有效性.     

Exploring casual point-and-tilt interactions for mobile geo-blogging

People record and share their experiences through text, audio and video. Increasingly they do this blogging from mobile devices. We illustrate a novel, mobile, low interaction cost approach to support the creation of a rich record of journeys made and places encountered: by pointing and tilting a mobile, users indicate their interests in a location. We built three mobile prototypes to explore the approach—the first one combines gestures and visual map feedback; the second provides a simpler visual interface; the third supports eyes-free interaction, allowing the user to simply point-and-tilt, with no visual display required. We describe two field studies undertaken to understand the value of the interaction styles afforded, then continue with a further user study to assess the interaction speed and accuracy between these interaction methods. We present the results of these studies and raise issues relevant to their design and to the wider class of devices and services concerned with mobile spatial information access.     

New interaction modes for rich panoramic live video experiences

The possibilities of panoramic video are based on the capabilities of high-resolution digital video streams and higher bandwidth's opportunities to broadcast, stream and transfer large content across platforms. With these opportunities also come challenges such as how to focus on sub-parts of the video stream and interact with the content shown on a large screen. In this paper, we present studies of two different interaction modes with a large-scale panoramic video for live experiences; we focus on interactional challenges and explore if it is (1) possible to develop new interactional methods/ways of approaching this type of high-resolution content and (2) feasible for users to interact with the content in these new ways. We developed prototypes for two different interaction modes: an individual system on a mobile device, either a tablet or a mobile phone, for interacting with the content on the same and a non-touch gesture-based system for the home or small group interaction. We present pilot studies where we explore the possibilities and challenges with these two interaction modes for panoramic content.     

Prototypes in pervasive computing

In product design, prototypes - approximations of a product along some dimensions - are the essential medium for information, interaction, integration, and collaboration. Information appliances such as mobile phones, digital cameras, and music players are a growing area of ubiquitous computing. Designers currently create two separate sets of prototypes: "looks-like" prototypes that show only the device's form (the atoms) and "works-like" prototypes that use a computer display to demonstrate the interaction (the bits). Because of the required skill and time investment, designers don't build comprehensive prototypes that join form and function until late in development.     

Perci: Pervasive Service Interaction with the Internet of Things

The advancement of ubiquitous computing technologies has greatly improved the availability of digital resources in the real world. Here, the authors investigate mobile interaction with tagged, everyday objects and associated information that's based on the Internet of Things and its technologies. Their framework for integrating Web services and mobile interaction with physical objects relies on information typing to increase interoperability. Two prototypes for mobile interaction with smart posters build upon this framework to realize multi-tag interaction with physical user interfaces. The authors' evaluation identifies usability issues regarding the design of physical mobile interactions, interfaces, and applications.     

Prototypes and paratypes: designing mobile and ubiquitous computing applications

Conducting user-centered design is a primary challenge in ubiquitous computing. Two kinds of prototyping techniques can be particularly effective for developing mobile and ubiquitous computing applications: compound prototypes and situated experience prototypes. Compound prototypes combine the final product's user interface (UI) with a computational implementation that runs on a separate system without the resource constraints of the target device. This model lets researchers and designers evaluate a new application's physicality and aesthetics as well as its functionality. Situated experience prototypes, or paratypes, include experimental protocols that attempt to reproduce user interaction with the system in real situations, and can optionally use common prototyping instruments such as paper mock-ups or physical props. Using our Personal Audio Loop (PAL) project as a case study, we discuss how you can fruitfully employ compound prototypes and paratypes when designing and developing mobile and ubiquitous computing applications.     

A Listening Keyboard for Users with Motor Impairments—A Usability Study

Computer users with motor impairments find it difficult and, in many cases, impossible to access PC functionality through the physical keyboard-and-mouse interface. Studies show that even able-bodied users experience similar difficulties when interacting with mobile devices; this is due to the reduced size/usability of the input interfaces. Advances in speech recognition have made it possible to design speech interfaces for alphanumeric data entry and indirect manipulation (cursor control). Although several related commercial applications exist, such systems do not provide a complete solution for arbitrary keyboard and mouse access, such as the access needed for, say, typing, compiling, and executing a C++ program.We carried out a usability study to support the development of a speech user interface for arbitrary keyboard access and mouse control. The study showed that speech interaction with an ideal listening keyboard is better for users with motor impairments than handstick, in terms of task completion time (37% better), typing rate (74% better), and error rates (63% better). We believe that these results apply to both permanent and task-induced motor impairments. In particular, a follow-up experiment showed that handstick approximates conventional modes of alphanumeric input available on mobile devices (e.g., PDAs, cellular phones, and personal organizers). These modes of input include miniaturized keyboards, stylus soft keyboards, cellular phone numberpads, and handwriting recognition software. This result suggests that a listening keyboard would be an effective mode for alphanumeric input on future mobile devices.This study contributed to the development of SUITEKeys—a speech user interface for arbitrary keyboard and mouse access available for MS platforms as freeware.     

Exploring multi-user interactions with dynamic NFC-displays

Near Field Communication (NFC) is an emerging technology for touch-based mobile interactions with single- and multi-tagged objects. Although the latter may allow for simultaneous and collaborative interactions, most prototypes were not designed for multiple users and were only evaluated with single-user interactions. In this paper, we investigate the design, usability and user experience of multi-user interactions on dynamic NFC-displays. These interactive surfaces use a grid of NFC-tags for the direct manipulation of projected application user interfaces. In two user studies with three prototypes for multi-user interaction, we evaluated the performance of dynamic NFC-displays, interactions among users and the interplay between mobile devices and large displays.     

基于Symbian的智能手机中文输入法的设计

开发面向智能手机的输入法是汉字输入技术的一个发展方向.首先分析了Symbian下的输入法接口以及相关技术,提出了一个客户机/服务器结构的输入法设计模式.然后给出了Symbian下输入法的设计要点,包括客户机端设计、服务器端设计以及服务器和客户机交互设计等内容.最后,简单阐述了码本的设计技术和检索机制.     

A formal model for plastic human computer interfaces

The considerable and significant progress achieved in the design and development of new interaction devices between man and machine has enabled the emergence of various powerful and efficient input and/or output devices. Each of these new devices brings specific interaction modes.With the emergence of these devices, new interaction techniques and modes arise and new interaction capabilities are offered. New user interfaces need to be designed or former ones need to evolve. The design of so called plastic user interfaces contributes to handling such evolutions. The key requirement for the design of such a user interface is that the new obtained user interface shall be adapted to the application and have, at least, the same behavior as the previous (adapted) one. This paper proposes to address the problem of user interface evolution due to the introduction of new interaction devices and/or new interaction modes. More, precisely, we are interested by the study of the design process of a user interface resulting from the evolution of a former user interface due to the introduction of new devices and/or new interaction capabilities. We consider that interface behaviors are described by labelled transition systems and comparison between user interfaces is handled by an extended definition of the bi-simulation relationship to compare user interface behaviors when interaction modes are replaced by new ones.     

Fuzzy algorithms for learning vector quantization

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.