User-defined gesture interaction for in-vehicle information systems

Multimedia Tools and Applications - Gesture elicitation study, a technique emerging from the field of participatory design, has been extensively applied in emerging interaction and sensing...     

User-defined gesture interaction for immersive VR shopping applications

Gesture elicitation studies, which are a popular technology for collecting requirements and expectations by involving real users in gesture design processes, often suffer from gesture disagreement and legacy bias and may not generate optimal gestures for a target system in practice. This paper reports a research project on user-defined gestures for interacting with immersive VR shopping applications. The main contribution of this work is the proposal of a more practical method for deriving more reliable gestures than traditional gesture elicitation studies. We applied this method to a VR shopping application and obtained empirical evidence for the benefits of deriving two gestures in the a priori stage and selecting the top-two gestures in the a posteriori stage of traditional elicitation studies for each referent. We hope that this research can help lay a theoretical foundation for freehand-gesture-based user interface design and be generalised to all freehand-gesture-based applications.     

Design of hand gesture recognition system for human-computer interaction

Multimedia Tools and Applications - Human-Computer interaction (HCI) with gesture recognition is designed to recognize a number of meaningful human expressions, and has become a valuable and...     

Adaptive visual gesture recognition for human–robot interaction using a knowledge-based software platform

In human–human communication we can adapt or learn new gestures or new users using intelligence and contextual information. Achieving natural gesture-based interaction between humans and robots, the system should be adaptable to new users, gestures and robot behaviors. This paper presents an adaptive visual gesture recognition method for human–robot interaction using a knowledge-based software platform. The system is capable of recognizing users, static gestures comprised of the face and hand poses, and dynamic gestures of face in motion. The system learns new users, poses using multi-cluster approach, and combines computer vision and knowledge-based approaches in order to adapt to new users, gestures and robot behaviors. In the proposed method, a frame-based knowledge model is defined for the person-centric gesture interpretation and human–robot interaction. It is implemented using the frame-based Software Platform for Agent and Knowledge Management (SPAK). The effectiveness of this method has been demonstrated by an experimental human–robot interaction system using a humanoid robot ‘Robovie’.     

Design methodology for battery powered embedded systems — In safety critical application

Battery powered embedded system can be considered as a power aware system for a safety critical application. There is a need of saving the battery power for such power aware system so that it can be used more efficiently, particularly in safety critical applications. Present paper describes power optimization procedure using real time scheduling technique having a specific dead line guided by the model based optimum current discharge profile of a battery. In any power aware system ‘energy optimization’ is one of the major issues for a faithful operation.     

Hand gesture guided robot-assisted surgery based on a direct augmented reality interface

Radiofrequency (RF) ablation is a good alternative to hepatic resection for treatment of liver tumors. However, accurate needle insertion requires precise hand-eye coordination and is also affected by the difficulty of RF needle navigation. This paper proposes a cooperative surgical robot system, guided by hand gestures and supported by an augmented reality (AR)-based surgical field, for robot-assisted percutaneous treatment. It establishes a robot-assisted natural AR guidance mechanism that incorporates the advantages of the following three aspects: AR visual guidance information, surgeon's experiences and accuracy of robotic surgery. A projector-based AR environment is directly overlaid on a patient to display preoperative and intraoperative information, while a mobile surgical robot system implements specified RF needle insertion plans. Natural hand gestures are used as an intuitive and robust method to interact with both the AR system and surgical robot. The proposed system was evaluated on a mannequin model. Experimental results demonstrated that hand gesture guidance was able to effectively guide the surgical robot, and the robot-assisted implementation was found to improve the accuracy of needle insertion. This human–robot cooperative mechanism is a promising approach for precise transcutaneous ablation therapy.     

Bayesian network model of maritime safety management

This paper presents a model of maritime safety management and its subareas. Furthermore, the paper links the safety management to the maritime traffic safety indicated by accident involvement, incidents reported by Vessel Traffic Service and the results from Port State Control inspections. Bayesian belief networks are applied as the modeling technique and the model parameters are based on expert elicitation and learning from historical data. The results from this new application domain of a Bayesian network based expert system suggest that, although several its subareas are functioning properly, the current status of the safety management on vessels navigating in the Finnish waters has room for improvement; the probability of zero poor safety management subareas is only 0.13. Furthermore, according to the model a good IT system for the safety management is the strongest safety-management related signal of an adequate overall safety management level. If no deficiencies have been discovered during a Port State Control inspection, the adequacy of the safety management is almost twice as probable as without knowledge on the inspection history. The resulted model could be applied to performing several safety management related queries and it thus provides support for maritime safety related decision making.     

Beat gesture recognition and finger motion control of a piano playing robot for affective interaction of the elderly

This paper introduces a piano playing robot in views of smart house and assistive robot technology to care the affective states of the elderly. We address the current issues in this research area and propose a piano playing robot as a solution. For affective interaction based on music, we first present a beat gesture recognition method to synchronize the tempo of a robot playing a piano with the desired tempo of the user. To estimate the period of an unstructured beat gesture expressed by any part of a body or an object, we apply an optical flow method, and use the trajectories of the center of gravity and normalized central moments of moving objects in images. In addition, we also apply a motion control method by which robotic fingers are trained to follow a set of trajectories. Since the ability to track the trajectories influences the sound a piano generates, we adopt an iterative learning control method to reduce the tracking error.     

基于多点手势识别的人机交互技术框架

提出了一种基于机器视觉的多点手势识别方法及其人机交互技术框架。指尖跟踪和手势识别服务程序通过一个普通的摄像机捕获用户手的运动,对多个指尖目标进行实时检测和跟踪,在指尖跟踪结果基础上利用BP神经网络实现多点手势识别,并根据指尖跟踪和手势识别结果构造相应的消息(包括低级指点消息和高级手势消息)发送给客户端应用程序,客户端响应消息并进行相应的处理。该框架可以帮助开发人员的在应用程序中增加类似iPhone多点触摸控制的多点手势识别控制功能,实现更加自然的人机交互,提高用户操作体验。     

一种在3D手势交互中模拟鼠标的方法

针对当下3D手势交互在执行2D交互任务时用户体验不佳,提出了一种模拟鼠标2D交互方式的3D手势交互技术。根据ISO9241-9标准,设计多方向点选实验,用于比较该3D手势交互技术、一般3D手势交互技术和日常鼠标这三种交互技术在执行典型2D交互任务时的标准吞吐量、指向-选择时间和错误率。以调查问卷的方式评估该交互技术的主观舒适性。结果显示,与一般的手势交互相比,在不降低吞吐率的情况下,该交互技术在执行2D交互任务时可以获得更高的准确率以及更好的舒适度体验。     

基于姿态识别的机器人人机交互系统设计

针对现有遥机器人人机交互系统依赖于视频信息使操控性能不稳定的问题,设计并实现一种基于姿态传感器的人机交互系统。利用姿态传感器获取人的姿态信息并对其进行解算,通过基于D‐S证据理论的信息融合法对姿态进行判定,使用语音确定动作指令的发送,对复杂动作进行组合,简化操作,通过“双虚拟模型”获取远端机器人的姿态,对运动轨迹进行修正。实验结果表明,该系统具有高稳定、高准确率及易操作的特点。     

A population randomization-based multi-objective genetic algorithm for gesture adaptation in human-robot interaction

In recent years,vision-based gesture adaptation has attracted great attention from many experts in the field of human-robot interaction,and many methods have be...     

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.     

Software health management: a necessity for safety critical systems

As software and software intensive systems are becoming increasingly ubiquitous, the impact of failures can be tremendous. In some industries such as aerospace, medical devices, or automotive, such failures can cost lives or endanger mission success. Software faults can arise due to the interaction between the software, the hardware, and the operating environment. Unanticipated environmental changes lead to software anomalies that may have significant impact on the overall success of the mission. Latent coding errors can at any time during system operation trigger faults despite the fact that usually a significant effort has been expended in verification and validation (V&V) of the software system. Nevertheless, it is becoming increasingly more apparent that pre-deployment V&V is not enough to guarantee that a complex software system meets all safety, security, and reliability requirements. Software Health Management (SWHM) is a new field that is concerned with the development of tools and technologies to enable automated detection, diagnosis, prediction, and mitigation of adverse events due to software anomalies, while the system is in operation. The prognostic capability of the SWHM to detect and diagnose failures before they happen will yield safer and more dependable systems for the future. This paper addresses the motivation, needs, and requirements of software health management as a new discipline and motivates the need for SWHM in safety critical applications.     

Immersive manipulation of virtual objects through glove-based hand gesture interaction

Immersive visualisation is increasingly being used for comprehensive and rapid analysis of objects in 3D and object dynamic behaviour in 4D. Challenges are therefore presented to provide natural user interaction to enable effortless virtual object manipulation. Presented in this paper is the development and evaluation of an immersive human?Ccomputer interaction system based on stereoscopic viewing and natural hand gestures. For the development, it is based on the integration of a back-projection stereoscopic system for object and hand display, a hybrid inertial and ultrasonic tracking system to provide the absolute positions and orientations of the user??s head and hands, as well as a pair of high degrees-of-freedom data gloves to provide the relative positions and orientations of digit joints and tips on both hands. For the evaluation, it is based on a two-object scene with a virtual cube and a CT (computed tomography) volume created for demonstration of real-time immersive object manipulation. The system is shown to provide a correct user view of objects and hands in 3D with depth, as well as to enable a user to use a number of simple hand gestures to perform basic object manipulation tasks involving selection, release, translation, rotation and scaling. Also included in the evaluation are some quantitative tests of the system performance in terms of speed and latency.     

Accelerometer-based gesture control for a design environment

Accelerometer-based gesture control is studied as a supplementary or an alternative interaction modality. Gesture commands freely trainable by the user can be used for controlling external devices with handheld wireless sensor unit. Two user studies are presented. The first study concerns finding gestures for controlling a design environment (Smart Design Studio), TV, VCR, and lighting. The results indicate that different people usually prefer different gestures for the same task, and hence it should be possible to personalise them. The second user study concerns evaluating the usefulness of the gesture modality compared to other interaction modalities for controlling a design environment. The other modalities were speech, RFID-based physical tangible objects, laser-tracked pen, and PDA stylus. The results suggest that gestures are a natural modality for certain tasks, and can augment other modalities. Gesture commands were found to be natural, especially for commands with spatial association in design environment control.     

Hand gesture recognition from depth and infrared Kinect data for CAVE applications interaction

This paper presents a real-time framework that combines depth data and infrared laser speckle pattern (ILSP) images, captured from a Kinect device, for static hand gesture recognition to interact with CAVE applications. At the startup of the system, background removal and hand position detection are performed using only the depth map. After that, tracking is started using the hand positions of the previous frames in order to seek for the hand centroid of the current one. The obtained point is used as a seed for a region growing algorithm to perform hand segmentation in the depth map. The result is a mask that will be used for hand segmentation in the ILSP frame sequence. Next, we apply motion restrictions for gesture spotting in order to mark each image as a ‘Gesture’ or ‘Non-Gesture’. The ILSP counterparts of the frames labeled as “Gesture” are enhanced by using mask subtraction, contrast stretching, median filter, and histogram equalization. The result is used as the input for the feature extraction using a scale invariant feature transform algorithm (SIFT), bag-of-visual-words construction and classification through a multi-class support vector machine (SVM) classifier. Finally, we build a grammar based on the hand gesture classes to convert the classification results in control commands for the CAVE application. The performed tests and comparisons show that the implemented plugin is an efficient solution. We achieve state-of-the-art recognition accuracy as well as efficient object manipulation in a virtual scene visualized in the CAVE.