The postural and control-display gain effects of distal pointing on upper extremity fatigue

Pointing at displays from a distance is becoming common in both work and domestic environments. Ray-casting interaction is easy for novices to learn and understand, but this technique can cause physiological fatigue. To address this issue, the present study aims to investigate the issue of fatigue caused by joint-based pointing methods and Control-Display gains (CD gains) via Fitts' task. Ten healthy subjects participated in the experiment and performed multi-directional tapping tests with three joint-based pointing methods and three CD gains. The experimental results indicated that the joint-based pointing methods indeed affected the physiological and subjective fatigue of the upper limb muscles and measured body parts during distal pointing tasks. The wrist-based pointing method, which can induce substantially lower physiological and subjective fatigue, appears to be superior to the other two methods. There were no significant main effects of CD gains on either physiological fatigue or subjective Borg's CR-10 rating.     

A Comparison of Head Pose and Deictic Pointing Interaction Methods for Smart Environments

Pointing is one of the more meaningful gestures in human expression. In this article, deictic and head pose pointing methods are evaluated as interaction means, with the objective of comparing their efficiency and delivered comfort, usability, and user experience to apply them to the deployment of services in smart spaces. Standard tools and questionnaires are applied in a user study with 20 individuals who have tried a Kinect V2-enabled pointing system to a) complete a multidirectional International Organization for Standardization (ISO) 9241-9 pointing task and b) test a service to command smart objects. The results show that both deictic and head pose interaction perform in a similar way, providing a throughput of 2.14 and 2.04 bits/s, respectively (comparable to joysticks or touchpads in equivalent ISO tasks). The system being quite robust to the user position in the Kinect coverage area, target size influences performance, but also learnability and the overall response when cursor visual cues are not available. From comfort, usability, and user experience responses, it can be said that deictic is perceived to perform better than head pose in terms of accuracy, efficiency, and organization, although head pose is preferred for its speed.     

Towards accurate cursorless pointing: the effects of ocular dominance and handedness

Pointing gestures are our natural way of referencing distant objects and thus widely used in HCI for controlling devices. Due to current pointing models’ inherent inaccuracies, most of the systems using pointing gestures so far rely on visual feedback showing users where they point at. However, in many environments, e.g., smart homes, it is rarely possible to display cursors since most devices do not contain a display. Therefore, we raise the question of how to facilitate accurate pointing-based interaction in a cursorless context. In this paper we present two user studies showing that previous cursorless techniques are rather inaccurate as they lack important considerations about users’ characteristics that would help in minimizing inaccuracy. We show that pointing accuracy could be significantly improved by acknowledging users’ handedness and ocular dominance. In a first user study (n=?33), we reveal the large effect of ocular dominance and handedness on human pointing behavior. Current ray-casting techniques neglect both ocular dominance and handedness as effects onto pointing behavior, precluding them from accurate cursorless selection. With a second user study (n=?25), we show that accounting for ocular dominance and handedness yields to significantly more accurate selections compared to two previously published ray-casting techniques. This speaks for the importance of considering users’ characteristics further to develop better selection techniques to foster more robust accurate selections.     

指势识别的实时指尖提取

指势行为识别作为一种理想的人机交互模式,而指势手指有效分割与指尖提取是关键。采用基于背景差分方法提取前景运动目标并消除背景影响,运用肤色分割方法,提取运动目标脸部、手部区域。在确定指势手与人脸位置关系的基础上,基于指势行为中指尖位于指势手的边沿轮廓,从指势手的外界矩形框与轮廓的交接点中定位指尖点。经实验证明该算法快速有效、提取精度高,且有一定的鲁棒性。     

Examining the costs of multiple trajectory pointing techniques

Pointing techniques that offer users multiple trajectories to a target have the potential to reduce pointing time by allowing a shorter than normal movement distance. However, such techniques potentially introduce additional elements into the pointing task involving identification of the alternative trajectories, assessment of their relative performance, and selection of the one to use. These additional tasks may reduce or negate the benefits of offering shorter paths. To better understand these issues, we developed a methodology for controlling the relative benefits of alternative target trajectories, and used it to evaluate three interfaces—a ‘pointer wrapping’ technique that allows the cursor to traverse from one screen edge to the opposing edge, and a system allowing users to choose between multiple cursors in two configurations (‘Ninja cursors’). We found that performance with these techniques was significantly worse than that predicted by Fitts' law for a single cursor, suggesting that the additional elements in their use are significant. Detailed analysis of behaviour during acquisitions showed that much of this cost is accrued in the mental preparation that precedes motor action, and in additional volatility in the pointing movements. We discuss how the method and findings may assist those developing enhanced pointing techniques.     

Evaluating gaze-based interface tools to facilitate point-and-select tasks with small targets

Gaze interaction affords hands-free control of computers. Pointing to and selecting small targets using gaze alone is difficult because of the limited accuracy of gaze pointing. This is the first experimental comparison of gaze-based interface tools for small-target (e.g. <12 × 12 pixels) point-and-select tasks. We conducted two experiments comparing the performance of dwell, magnification and zoom methods in point-and-select tasks with small targets in single- and multiple-target layouts. Both magnification and zoom showed higher hit rates than dwell. Hit rates were higher when using magnification than when using zoom, but total pointing times were shorter using zoom. Furthermore, participants perceived magnification as more fatiguing than zoom. The higher accuracy of magnification makes it preferable when interacting with small targets. Our findings may guide the development of interface tools to facilitate access to mainstream interfaces for people with motor disabilities and other users in need of hands-free interaction.     

Analysis and comparison of target assistance techniques for relative ray-cast pointing

Pointing at displays from a distance is becoming a common method of interacting with computer applications and entertainment systems, using devices such as the Wii Remote, the PlayStation Move controller, or the Microsoft Kinect. These systems often implement relative forms of ray-cast pointing, in which the user simply points a hand-held input device towards targets on the screen. Ray-casting interaction is easy for novices to learn and understand, but this technique often suffers from accuracy problems: for example, hand jitter, arm fatigue, calibration drift, or lack of skill can all reduce people’s ability to acquire and select on-screen targets. In this paper, we analyse and evaluate the idea of target assistance as a way to address the accuracy problems of ray-cast pointing. Although several assistance schemes have been proposed for mouse-based pointing, these ideas have not been tested in distant-pointing settings, and there is little knowledge available to guide design in this increasingly common interaction scenario. To establish this basic design knowledge, we carried out four studies of relative ray-casting using three different target assistance techniques—two motor-space techniques (sticky targets and a novel form of target gravity), and one acquisition-feedback technique that combined visual, tactile, and auditory feedback. Our first three studies tested each assistance technique separately, to explore how different parameters for each method affected performance and perceptibility. Our fourth study carried out a direct comparison of the best versions of each technique, and also examined the effects of distractor objects placed in the path to the target. Our studies found that the two motor-space techniques were extremely effective in improving selection accuracy without being highly obvious to users, and that the new gravity-based technique (which attracts the cursor even when it is not over the target) performed best of all. There was no observed effect on performance when the combined acquisition-feedback technique was used. Our studies are the first to comprehensively explore the optimization, performance, and perceptibility of target assistance techniques for relative ray-casting—our results provide designers with clear guidelines about what methods to use, how to configure the techniques, and what effects can be expected from their use.     

Using tangible user interfaces in computer-based training systems for low-functioning autistic children

In this paper, the design of a computer-based training (CBT) system for low-functioning autistic children is addressed. The emphasis is on ease-of-use and learning efficiency of CBT systems with different interaction styles, namely the WIMP (Window Icon Menu Pointing Device) and TUI (Tangible User Interface) interaction styles. Two WIMP-based CBT systems with different pointing devices were involved in the study. The first system applied a standard computer mouse as a pointing device, while the second one employed a touch screen instead. For the TUI-based CBT system, a tabletop setting was adopted. Based on the known characteristics of TUI and children with autism, as well as related cognitive and learning theories, the benefits of TUI for low-functioning autistic children have been investigated. Elementary skill teaching was chosen as a case study for performance evaluation of these CBT systems. Empirical results show that the touch-based and TUI-based systems offered much better ease-of-use performance than that of the mouse-based system. Regarding learning efficacy, experimental results show that the TUI-based system achieved higher skill improvement, as compared with the WIMP-based system and a non-computer training method. Some guidelines and suggestions for the design of a TUI-based system for children with autism are summarized.     

Input devices for web browsing: age and hand effects

The work reported in this paper examined performance on a mixed pointing and data entry task using direct and indirect positioning devices for younger, middle-aged, and older adults (n=72) who were experienced mouse users. Participants used both preferred and non-preferred hands to perform an item selection and text entry task simulating a typical web page interaction. Older adults performed more slowly than middle-aged adults who in turn performed more slowly than young adults. Performance efficiency was superior with the mouse for older adults only on the first two trial blocks. Thereafter mouse and light pen yielded equivalent performance. For other age groups, mouse and light pen were equivalent at all points of practice. Contrary to prior research revealing superior performance with a light pen for pure pointing tasks, these results suggest that older adults may initially perform worse with a light pen than a mouse for mixed tasks.     

Assisting Visually Impaired People to Acquire Targets on a Large Wall-Mounted Display

Large displays have become ubiquitous in our everyday lives, but these displays are designed for sighted people.This paper addresses the need for visually impaired people to access targets on large wall-mounted displays. We developed an assistive interface which exploits mid-air gesture input and haptic feedback, and examined its potential for pointing and steering tasks in human computer interaction（HCI）. In two experiments, blind and blindfolded users performed target acquisition tasks using mid-air gestures and two different kinds of feedback（i.e., haptic feedback and audio feedback）. Our results show that participants perform faster in Fitts＇ law pointing tasks using the haptic feedback interface rather than the audio feedback interface. Furthermore, a regression analysis between movement time（MT） and the index of difficulty（ID）demonstrates that the Fitts＇ law model and the steering law model are both effective for the evaluation of assistive interfaces for the blind. Our work and findings will serve as an initial step to assist visually impaired people to easily access required information on large public displays using haptic interfaces.     

Speed–accuracy tradeoff in Fitts’ law tasks—on the equivalency of actual and nominal pointing precision

Pointing tasks in human–computer interaction obey certain speed–accuracy tradeoff rules. In general, the more accurate the task to be accomplished, the longer it takes and vice versa. Fitts’ law models the speed–accuracy tradeoff effect in pointing as imposed by the task parameters, through Fitts’ index of difficulty (I_d) based on the ratio of the nominal movement distance and the size of the target. Operating with different speed or accuracy biases, performers may utilize more or less area than the target specifies, introducing another subjective layer of speed–accuracy tradeoff relative to the task specification. A conventional approach to overcome the impact of the subjective layer of speed–accuracy tradeoff is to use the a posteriori “effective” pointing precision W_e in lieu of the nominal target width W. Such an approach has lacked a theoretical or empirical foundation. This study investigates the nature and the relationship of the two layers of speed–accuracy tradeoff by systematically controlling both I_d and the index of target utilization I_u in a set of four experiments. Their results show that the impacts of the two layers of speed–accuracy tradeoff are not fundamentally equivalent. The use of W_e could indeed compensate for the difference in target utilization, but not completely. More logical Fitts’ law parameter estimates can be obtained by the W_e adjustment, although its use also lowers the correlation between pointing time and the index of difficulty. The study also shows the complex interaction effect between I_d and I_u, suggesting that a simple and complete model accommodating both layers of speed–accuracy tradeoff may not exist.     

基于自适应光标的图形用户界面输入效率优化

提高图形用户界面(graphical user interface)的输入效率,是人机交互中的一项重要研究内容.已有的研究包括点击增强技术和自适应界面技术,前者改变光标的控制方式或呈现方式,后者改变界面上控件的位置布局,但两种技术都存在不足.通过分析界面操作,提出了图形用户界面输入效率的评价模型;然后,在此基础上提出一种人机交互效率优化技术:自适应光标.它以自适应的方式,有选择地对界面上用户可能访问的控件通过点击增强技术支持,实现快速访问.该方法既解决了以往的自适应界面技术因频繁调整控件布局而给用户带来额外认知成本的问题,也克服了点击增强技术仅适用于稀疏控件布局的限制.为了检验其可用性,在控件较多的Visual Studio上实现了自适应光标技术.实验结果表明,使用自适应光标技术可以将获取目标的时间缩短27.7％,显著提高了图形用户界面的输入效率.     

A human motor behavior model for distal pointing tasks

Models of human motor behavior are well known as an aid in the design of user interfaces (UIs). Most current models apply primarily to desktop interaction, but with the development of non-desktop UIs, new types of motor behaviors need to be modeled. Distal pointing—pointing directly at a target that is remotely situated with respect to the input device—is such a motor behavior. A model of distal pointing would be particularly useful in the comparison of different interaction techniques, because the performance of such techniques is highly dependent on user strategy, making controlled studies difficult to perform. Inspired by Fitts’ law, we studied four possible models and concluded that movement time for a distal pointing task is best described as a function of the angular amplitude of movement and the angular size of the target. Contrary to Fitts’ law, our model shows that the angular size has a much larger effect on movement time than the angular amplitude and that the growth in the difficulty of the tasks is quadratic, rather than linear. We estimated the model's parameters experimentally with a correlation coefficient of 96%.     

On the representation and management of medical records in a knowledge-based system

The need for adapting Hospital Information Systems (HISs) to new applications (aimed at clinical data management) and to the ever growing population requires an evolution from both the representation and the usability points of view. The goal of this paper is to discuss the interdependencies of the advanced data representation models necessary to the new HISs and the interaction requirements. In particular, we present the technical taken in the MILORD (Multimedia Interaction with Large Object-oriented Radiological and clinical Databases) project to implement a departmental environment which integrates different kinds of clinical data in a user-friendly and homogeneous way. The implementation platform of the MILORD system is based on results obtained in the KIWIS project, which produced an advanced knowledge-base environment for large database systems. The interaction with a HIS is analyzed identifying the HIS-specific interaction tasks, the typical phases of the interaction with an information system, and the necessary interaction paradigms. In particular we describe the approach adopted in KIWIS for overcoming the problem of querying object-oriented databases and we discuss the new requirements that multimediality poses on the interaction paradigms.     

Design and feasibility study of an integrated pointing device apparatus for individuals with spinal cord injury

Despite the commercial availability of numerous computer-pointing devices, many severely disabled individuals still rely on customized equipment to operate computers. This study presents a novel Integrated Pointing Device Apparatus (IPDA) that integrates numerous commercial pointing devices. The novel IPDA, which complies with a standard USB 1.1 interface, is compatible with most tested computer-pointing devices and flexibly integrates commercial computer devices, tailoring them to suit individual needs. By using simple integrated circuit design and low-cost electronic components, this low-cost apparatus is easily maintained. The feasibility of the IPDA was evaluated by four subjects with high-level cervical (C4-5) spinal cord injury (SCI). Participants performed normal move-and-click and drag-and-drop tasks typically performed by computer pointing devices. Each participant not able to use a traditional computer mouse or trackball were able to operate a computer adequately with the IPDA and three including one operating a trackball with his chin, operated computers easily and smoothly. This feasibility study showed that the IPDA effectively integrates commercial pointing devices, thereby providing the possibility for some people with SCI to obtain computer operability. This study demonstrated the advantages of flexibility, low cost, and acceptable efficiency of the novel IPDA.     

A switched dynamic model for pointing tasks with a computer mouse

In this paper, we study one of the most fundamental tasks in human‐computer interaction, the pointing task. It can be described simply as reaching a target with a cursor starting from an initial position (e.g. executing a movement using a computer mouse to select an icon). In this paper, a switched dynamic model is proposed to handle cursor movements in indirect pointing tasks. The model contains a ballistic movement phase governed by a nonlinear model in Lurie form and a corrective movement phase described by a linear visual‐feedback system. The stability of the model is first established and the derived model is then validated with experimental data acquired in a pointing task with a mouse. It is established that the measured data of pointing movements of different types can be fitted within the proposed model. Numerical comparison against pointing models available in the literature is also provided.     

A haptic-based approach to virtual training for aerospace industry

In the last years, the industrial world has been increasingly adopting computer-aided solutions for design for maintainability and maintenance training tasks with the goal to reduce development costs and to shorten time, and to improve product and service quality. Computer-based training systems created to simulate machine assembly maintenance are normally operated by means of ordinary human–computer interfaces (keyboard, mouse, etc.), but this usually results in systems that are far from the real procedures, and therefore not effective in terms of training. In this study, we show that a better solution may come from the combination of virtual reality techniques and haptic interaction. To this regard, we present the results of a research aimed at testing and evaluating the effectiveness of the haptic feedback for first-person maintenance tasks targeted to the aerospace industry. The proposed system implements an interaction environment in which each of the main maintenance activities can be simulated by the trainee exploiting a hand-based commercial haptic device, operated by means of specific haptic-rendering techniques to provide realistic feedbacks during manipulation. A usability study is included to help assessing the potential of this approach.     

Air pointing: Design and evaluation of spatial target acquisition with and without visual feedback

Sensing technologies such as inertia tracking and computer vision enable spatial interactions where users make selections by ‘air pointing’: moving a limb, finger, or device to a specific spatial region. In addition of expanding the vocabulary of possible interactions available, air pointing brings the potential benefit of enabling ‘eyes-free’ interactions, where users rely on proprioception and kinaesthesia rather than vision. This paper explores the design space for air pointing interactions, and presents tangible results in the form of a framework that helps designers understand input dimensions and resulting interaction qualities. The framework provides a set of fundamental concepts that aid in thinking about the air pointing domain, in characterizing and comparing existing solutions, and in evaluating novel techniques. We carry out an initial investigation to demonstrate the concepts of the framework by designing and comparing three air pointing techniques: one based on small angular ‘raycasting’ movements, one on large movements across a 2D plane, and one on movements in a 3D volume. Results show that large movements on the 2D plane are both rapid (selection times under 1 s) and accurate, even without visual feedback. Raycasting is rapid but inaccurate, and the 3D volume is expressive but slow, inaccurate, and effortful. Many other findings emerge, such as selection point ‘drift’ in the absence of feedback. These results and the organising framework provide a foundation for innovation and understanding of air pointing interaction.