Toward multimodal human-computer interface

Recent advances in various signal processing technologies, coupled with an explosion in the available computing power, have given rise to a number of novel human-computer interaction (HCI) modalities: speech, vision-based gesture recognition, eye tracking, electroencephalograph, etc. Successful embodiment of these modalities into an interface has the potential of easing the HCI bottleneck that has become noticeable with the advances in computing and communication. It has also become increasingly evident that the difficulties encountered in the analysis and interpretation of individual sensing modalities may be overcome by integrating them into a multimodal human-computer interface. We examine several promising directions toward achieving multimodal HCI. We consider some of the emerging novel input modalities for HCI and the fundamental issues in integrating them at various levels, from early signal level to intermediate feature level to late decision level. We discuss the different computational approaches that may be applied at the different levels of modality integration. We also briefly review several demonstrated multimodal HCI systems and applications. Despite all the recent developments, it is clear that further research is needed for interpreting and fitting multiple sensing modalities in the context of HCI. This research can benefit from many disparate fields of study that increase our understanding of the different human communication modalities and their potential role in HCI     

Toward more compatible human-computer interfaces

Some poorly understood aspects of human-computer interfaces are identified. The use of guided evolution, or interactive design, to guarantee design flexibility by intentionally leaving some system options open during the early stages is discussed. The importance of evaluating a product design as it is evolving and funneling the results constantly back into the design is stressed, and some methods for doing so are briefly described. Principles recognized by system developers as applicable to interface design in general, independent of the anticipated end use, are presented. The need to design interfaces that meet the needs of both novice and experienced users is considered. Speculations on future systems are offered     

Emotion recognition in human-computer interaction

Two channels have been distinguished in human interaction: one transmits explicit messages, which may be about anything or nothing; the other transmits implicit messages about the speakers themselves. Both linguistics and technology have invested enormous efforts in understanding the first, explicit channel, but the second is not as well understood. Understanding the other party's emotions is one of the key tasks associated with the second, implicit channel. To tackle that task, signal processing and analysis techniques have to be developed, while, at the same time, consolidating psychological and linguistic analyses of emotion. This article examines basic issues in those areas. It is motivated by the PKYSTA project, in which we aim to develop a hybrid system capable of using information from faces and voices to recognize people's emotions     

一种新型复合人机输入平台

为解决传统计算机周边输入设备占用空间大、使用不方便、定位不精确等问题,构建一种以电容式多指触摸屏为基础的新型复合人机输入平台,并通过软件设计集成鼠标功能、键盘功能和触摸功能于一体;介绍复合人机输入平台的硬件结构和软件构件,说明复合设备的模式切换方法,分别给出三种工作模式正常工作的结果;简化了计算机周边设备,使人机交互更...     

Enhancing human-computer interaction in medical segmentation

Medical image acquisition devices are becoming increasingly multidimensional, and predictions assume 5000 images per patient study within the next decade. Therefore, new paradigms for computerized medical image analysis and visualization are of fundamental importance to make possibly full use of the information buried in the enormous flood of image data. The aim of the presented project is the optimal cooperation between computer-based image analysis algorithms and human operators using new closed-loop segmentation systems for improved information flow. This paper describes an enhanced, visuo-haptic interaction tool we have developed for medical segmentation. Evaluation studies with the system, which confirm the value of adding haptic feedback, are also presented.     

基于人眼注视非穿戴自然人机交互

提出了一种基于人眼注视的非穿戴自然人机交互新方法。基于人类生物结构特征,采用主动形状模型确定人眼轮廓特征点,并根据HSV色彩空间中各色彩和亮度信息,构建综合反映人眼纹理及其空间位置的人眼特征直方图,采用粒子滤波方法,对人眼目标进行跟踪与定位,在克服光照和人脸姿态变化影响的同时,获取稳定的人眼轮廓特征点,满足人机交互的实效性要求。基于最大三角化划分人眼轮廓特征,构建人眼几何模型并确定人眼注视方向,通过图像帧间的均值滤波,确定人眼注视交互目标,用户无需佩带任何设备或有助于识别的辅助标志,实现非穿戴的自然人机交互。在用户灵活交互的同时,满足人机交互的舒适性和自由性要求。通过实验对比,验证了本文方法有效、可行。     

GIS人机交互的进展与挑战

人机交互研究旨在提供一种高效的人-机通信方式,使用户操作更加便捷。随着计算机软、硬件技术与地理信息技术的发展,公众参与型GIS成为GIS发展的趋势和潮流。总结了GIS人机交互的发展过程与现状,论述了GIS人机交互涉及的主要问题,指出了其中存在的不足,并提出了一些解决思路。在此基础上,结合当前技术现状对GIS人机交互的发展趋势进行了展望。     

Audio-visual interaction in multimodal communication

Multimedia signal processing is more than simply “putting together” text, audio, images, and video. It is the integration and interaction among these different media that creates new systems and new research challenges and opportunities. In multimodal communication where speech is involved, audio-visual interaction is particularly significant     

基于视线的眼动跟踪人机交互系统研究

介绍了系统的工作原理,建立了信息交流和控制的系统结构,将面向对象和基于事件驱动的处理方法应用于通信协议的建立和通信控制过程.结合具体通信控制单元三菱PLC和FX1N-232-BD与PC串口RS-232C的通信,编写了算法和通信控制的代码.     

Lurking: an underestimated human-computer phenomenon

In this article, the authors offer their perspective about online learning. In general, we know that online learning develops through interaction and that it's a collaborative process where students actively engage in writing and reading messages among themselves and with the instructor. However, it's also well known that in any online community, not all users are equally active, and there are indeed people who never take an active part-the so-called lurkers. This article focuses on the lurkers; the authors ran extensive experiments to demonstrate whether there's a relationship between the writing and reading behavior of online students and whether active participation influences learning efficiency. An interesting related result that emerged from the study is that the effort of the instructor in terms of reading and writing posts is higher than that of the learners themselves.     

基于人机交互界面的碎纸片拼接复原

2013年的全国大学生数学建模竞赛提出了碎纸片的拼接复原的问题,然而绝大多数论文都先设置了不同筛选条件对碎片进行筛选得到最有可能处于同一行的碎片,然后再逐行对得到的碎片进行手工调整得到完整的还原图,笔者认为这种方法效率不是很高并且存在一些难以克服的精度问题,从而在本文中给出了一种通过人机交互界面的优化模型来实现碎片复原的思路和算法,其复原效率可达到90%或95%以上。     

基于人机交互的工业产品设计模型研究

针对传统产品设计模型一直存在设计不准确的问题,提出基于人机交互的工业产品设计模型研究。对工业产品模型结构与人机交互技术进行了融合设计。人机交互技术对数据传递以及传感反馈系统要求较高,因此进行结构的优化过程中对结构组成以及结构顺序进行了重新编排。在对人机交互工业产品模型的建立过程中,对人机交互工业产品模型矩阵进行了系统设计,保证数据反馈能力以及工业产品实际质量,并且避免发生由于感知问题产生工业产品不合格现象,引入等级映射计算,保证数据交互畅通以及数据传感有效性,极大提高机械生产过程的感知力,模型还设计了工业产品检验机制,保证工业生产有效性。实验结果证明,设计的模型能够有效地应用于工业生产之中。     

人机交互安全攻防综述

随着物联网和人工智能等技术的发展,人机交互应用也在不断推陈出新。然而,人机交互技术的发展也伴生新的安全问题,人机交互安全攻击事件频发。综述分析了 4 种人机交互技术——HID-USB、指纹识别、语音识别和人脸识别的安全攻防现状及攻防效益,并探讨未来人机交互安全的研究方向,同时给出了一些建议。     

面向任务的空基信息系统用户化人机交互设计

根据任务目标要求和执行过程,用户化定制空基信息系统人机交互界面和操作流程,使得系统人机交互功能更贴近于用户业务需求,更有效地协助用户实现其任务目标。具体方法是,以任务执行过程为主线,按照信息的获取、加工、存储、应用、分发等处理流程将任务分解为多个子任务,并以此为参照,将空基信息系统细化分解为多个子系统,然后根据各个子系统的操作要求定义系统操作员席位职责。一方面,面向任务目标要求和执行过程的系统功能分解方法确保了用户需求的落实;另一方面,多用户并行处理模式简化了用户操作视图,增强了系统的易用性和人机交互效率。该方法不仅可以在新系统研制过程中发挥良好作用,还可以在已有系统改造项目中得以应用。     

基于虚拟现实的人机交互下协同式产品外观设计

为了解决当前产品外观设计方法很难完全模拟人类智能对产品外观进行设计的问题,提出一种新的基于虚拟现实的人机交互下协同式产品外观设计方法,给出虚拟现实设计环境总体结构,在虚拟环境下,计算机主要负责对符合约束的方案进行求解,而设计者主要负责确定产品外观设计过程中的变量、约束条件及设计目标等,令二者协同设计产品外观,获取最佳设计方案。为了更好地实现人机交互,对设计者知识描述进行转换,依据创新元对设计者决策信息进行分类、描述以及保存。通过再次转换将参数确定问题变成基本参数描述问题,以确定设计参数。实验结果表明,所提方法鲁棒性和多样性高,且设计的产品外观普遍评价较高。     

人机交互系统的实时人眼跟踪技术研究

为了解决面向移动设备的眼动跟踪技术存在的计算效率低、跟踪误差大等问题,采用改进眼动跟踪技术的方法,开展人机交互系统的研究。主要内容包含系统总体架构、系统用户登录、系统功能实现等内容设计,并对系统功能实现展开分析。研究结果表明,系统各功能及界面实现均满足用户要求,可用于指导新手用户对重要信息的浏览,提升用户理解效率。     

电子产品设计中人机交互的应用研究

在智能化时代的背景下,各式各样的电子产品也随之问世.虽然电子产品越来越多,但是单一的交流方式也变得不能满足人们的需要,所以电子产品设计越来越注重人机交互.人机交互的应用让电子产品设计又有了一个新的高度.本论文通过对电子产品设计的研究,分析人机交互在电子产品设计中的应用.     

Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction

This paper describes technology and tools for intelligent human-computer interaction (IHCI) in which human cognitive, perceptual, motor and affective factors are modeled and used to adapt the H-C interface. IHCI emphasizes that human behavior encompasses both apparent human behavior and the hidden mental state behind behavioral performance. IHCI expands on the interpretation of human activities, known as W4 (what, where, when, who). While W4 only addresses the apparent perceptual aspect of human behavior the W5+ technology for IHCI described in this paper addresses also the why and how questions, whose solution requires recognizing specific cognitive states. IHCI integrates parsing and interpretation of nonverbal information with a computational cognitive model of the user which, in turn, feeds into processes that adapt the interface to enhance operator performance and provide for rational decision-making. The technology proposed is based on a general four-stage interactive framework, which moves from parsing the raw sensory-motor input, to interpreting the user's motions and emotions, to building an understanding of the user's current cognitive state. It then diagnoses various problems in the situation and adapts the interface appropriately. The interactive component of the system improves processing at each stage. Examples of perceptual, behavioral, and cognitive tools are described throughout the paper Adaptive and intelligent HCI are important for novel applications of computing, including ubiquitous and human-centered computing.