Use of force plate instrumentation to assess kinetic variables during touch screen use

Touch screens are becoming ubiquitous technology, allowing for enhanced speed and convenience of user interfaces. To date, the majority of touch screen usability studies have focused on timing and accuracy of young, healthy individuals. This information alone may not be sufficient to improve accessibility and usability of touch screens. Kinetic data (e.g. force, impulse, and direction) may provide valuable information regarding human performance during touch screen use. Since kinetic information cannot be measured with a touch screen alone, touch screen-force plate instrumentation, software, and methodology were developed. Individuals with motor control disabilities (Cerebral Palsy and Multiple Sclerosis), as well as gender- and age-matched non-disabled participants, completed a pilot reciprocal tapping task to evaluate the validity of this new instrumentation to quantify touch characteristics. Results indicate that the instrumentation was able to successfully evaluate performance and kinetic characteristics. The kinetic information measured by the new instrumentation provides important insight into touch characteristics which may lead to improved usability and accessibility of touch screens.     

Eyes-free interaction with free-hand gestures and auditory menus

Auditory interfaces can overcome visual interfaces when a primary task, such as driving, competes for the attention of a user controlling a device, such as radio. In emerging interfaces enabled by camera tracking, auditory displays may also provide viable alternatives to visual displays. This paper presents a user study of interoperable auditory and visual menus, in which control gestures remain the same in the visual and the auditory domain. Tested control methods included a novel free-hand gesture interaction with camera-based tracking, and touch screen interaction with a tablet. The task of the participants was to select numbers from a visual or an auditory menu including a circular layout and a numeric keypad layout. Results show, that even with participant's full attention to the task, the performance and accuracy of the auditory interface are the same or even slightly better than the visual when controlled with free-hand gestures. The auditory menu was measured to be slower in touch screen interaction, but questionnaire revealed that over half of the participants felt that the circular auditory menu was faster than the visual menu. Furthermore, visual and auditory feedback in touch screen interaction with numeric layout was measured fastest, touch screen with circular menu second fastest, and the free-hand gesture interface was slowest. The results suggest that auditory menus can potentially provide a fast and desirable interface to control devices with free-hand gestures.     

Touch a screen or turn a knob: choosing the best device for the job

Input devices enable users to interact with systems. In two experiments, we assessed whether and how task demands and user age influenced task performance for a direct input device (touch screen) and an indirect input device (rotary encoder). In Experiment 1, 40 younger (18-28 years) and 40 middle-aged to older adults (51-65 years) performed tasks using controls such as sliders, up/down buttons, list boxes, and text boxes while using a system. The optimal input device to facilitate performance was dependent on the task being performed and the age of the user. In Experiment 2, touch screen use was assessed for 20 younger (19-23 years) and 20 older adults (51-70 years). Task demands were manipulated through button size, movement distance, direction, and type of movement. Performance was moderated by the age of the user and by task demands. Actual or potential applications of this research include guidance for the optimal selection of input devices for different user populations and task characteristics.     

Touchable area: An empirical study on design approach considering perception size and touch input behavior

As the use of mobile touch devices continues to increase, distinctive user experiences can be provided through a direct manipulation. Therefore, the characteristics of touch interfaces should be considered regarding their controllability. This study aims to provide a design approach for touch-based user interfaces. A derivation procedure for the touchable area is proposed as a design guideline based on input behavior. To these ends, two empirical tests were conducted through a smart phone interface. Fifty-five participants were asked to perform a series of input tasks on a screen. As results, touchable area with a desirable hit rate of 90% could be yielded depending on the icon design. To improve the applicability of the touchable area, user error was analyzed based on omission-commission classification. The most suitable design had a hit rate of 95% compared to 90 and 99%. This study contributes practical implications for user interaction design with finger-based controls.Relevance to industryThis research describes a distinctive design approach that guarantees the desired touch accuracy for effective use of mobile touch devices. Therefore, the results will encourage interface designers to take into account the input behavior of fingers from a user-centered perspective.     

Investigation of Icon Design and Touchable Area for Effective Smart Phone Controls

As the full‐touch screen is being implemented in more smart phones, controllability of touch icons need to be considered. Previous research focused on recommendations for absolute key size. However, the size of tactual input on touch interface is not precisely equal to the icon size. This study aims to determine the suitable touchable area to improve touch accuracy. In addition, there was an investigation into the effect of layout (3 × 4, 4 × 5, 5 × 6, and 6 × 8) and icon ratio (0.5, 0.7, and 0.9). To achieve these goals, 40 participants performed a set of serial tasks on the smart phone. Results revealed that the layout and icon ratio were statistically significant on the user response: input offset, hit rate, task completion time, and preference. The 3 × 4 and 4 × 5 layouts were shown to have better performance. The icon ratio of 0.9 was shown to have greater preference. Furthermore, the hit rate (proportion of correct input) of touchable area was estimated through the bivariate normal distribution of input offset. The hit rate could vary, depending on the size of touchable area, which is a rectangle that yields a specific hit rate. A derivation procedure of the touchable area was proposed to guarantee the desirable hit rate. Meanwhile, the locations of the central region indicated a pattern of vertical touch and showed better performance. The users felt more difficulty when approaching the edge of the frame. The results of this study could be used in the design of touch interfaces for mobile devices.     

A comparison of two contemporary types of in-car multifunctional interfaces

A driving simulator study was conducted to investigate the effects of carrying out a variety of tasks using two different types of contemporary in-car multifunctional interfaces: a touch screen interface and an interface manoeuvred by a rotary control. Participants drove on a curved rural road while performing tasks such as list scrolling, radio tuning, alphanumeric input and continuous adjustments. The results indicate that, in terms of task completion time and the number of glances made to the display, the optimal interface is dependent on the task being performed. The touch screen interface was better for alphanumeric input tasks and the interface manoeuvred by a rotary control was better for continuous adjustments and list scrolling. Alphanumeric input seems to be more demanding than other tasks, independent of the interface used. It was apparent in this simulator study that both interfaces affected the lateral control performance, but lateral control performance deteriorated to a greater extent when the touch screen interface was used, probably partially as a result of the lower display position.     

Two-mode target selection: Considering target layouts in small touch screen devices

A variety of studies have been conducted to improve methods of selecting a tiny virtual target on small touch screen interfaces of handheld devices such as mobile phones and PDAs. These studies, however, focused on a specific selection method, and did not consider various layouts resulting from different target sizes and densities on the screen. This study proposes a Two-Mode Target Selection (TMTS) method that automatically detects the target layout and changes to an appropriate mode using the concept of an activation area. The usability of TMTS was compared experimentally to those of other methods. TMTS changed to the appropriate mode successfully for a given target layout and showed the shortest task completion time and the fewest touch inputs. TMTS was also rated by the users as the easiest to use and the most preferred. TMTS could significantly increase the ease, accuracy, and efficiency of target selection, and thus enhance user satisfaction when the users select targets on small touch screen devices.

Relevance to Industry

The results of this study can be used to develop fast and accurate target selection methods in handheld devices with touch screen interfaces especially when the users use their thumb to activate the desired target.     

Haptic force-feedback devices for the office computer: performance and musculoskeletal loading issues

Pointing devices, essential input tools for the graphical user interface (GUI) of desktop computers, require precise motor control and dexterity to use. Haptic force-feedback devices provide the human operator with tactile cues, adding the sense of touch to existing visual and auditory interfaces. However, the performance enhancements, comfort, and possible musculoskeletal loading of using a force-feedback device in an office environment are unknown. Hypothesizing that the time to perform a task and the self-reported pain and discomfort of the task improve with the addition of force feedback, 26 people ranging in age from 22 to 44 years performed a point-and-click task 540 times with and without an attractive force field surrounding the desired target. The point-and-click movements were approximately 25% faster with the addition of force feedback (paired t-tests, p < 0.001). Perceived user discomfort and pain, as measured through a questionnaire, were also smaller with the addition of force feedback (p < 0.001). However, this difference decreased as additional distracting force fields were added to the task environment, simulating a more realistic work situation. These results suggest that for a given task, use of a force-feedback device improves performance, and potentially reduces musculoskeletal loading during mouse use. Actual or potential applications of this research include human-computer interface design, specifically that of the pointing device extensively used for the graphical user interface.     

To twist,roll, stroke or poke? A study of input devices for menu navigation in the cockpit

Modern interfaces within the aircraft cockpit integrate many flight management system (FMS) functions into a single system. The success of a user's interaction with an interface depends upon the optimisation between the input device, tasks and environment within which the system is used. In this study, four input devices were evaluated using a range of Human Factors methods, in order to assess aspects of usability including task interaction times, error rates, workload, subjective usability and physical discomfort. The performance of the four input devices was compared using a holistic approach and the findings showed that no single input device produced consistently high performance scores across all of the variables evaluated. The touch screen produced the highest number of ‘best’ scores; however, discomfort ratings for this device were high, suggesting that it is not an ideal solution as both physical and cognitive aspects of performance must be accounted for in design.

Practitioner summary: This study evaluated four input devices for control of a screen-based flight management system. A holistic approach was used to evaluate both cognitive and physical performance. Performance varied across the dependent variables and between the devices; however, the touch screen produced the largest number of ‘best’ scores.     

The influence of image interactivity upon user engagement when using mobile touch screens

Touch screens are a key component of consumer mobile devices such as smartphones and tablets, as well as an increasingly common self-service component of information retrieval on fixed screens and mobile devices in-store. The ubiquity of touch screens in daily life increases consumer accessibility and extended use for shopping, whilst software innovations have increased the functionality of touch screens, for example the extent to which images respond to fingertip control. This study examines how users engage with interactive visual rotation and tactile simulation features while browsing fashion clothing products on touch screen devices and thus contributes to retail touch screen research that previously focused on in-store kiosks and window displays. Findings show that three dimensions of user engagement (endurability, novelty and felt involvement) are positively influenced by both forms of manipulation. In order to examine the extent to which touch screen user engagement varies with individual preferences for an in-store experience, the paper also examines whether user engagement outcomes are mediated by an individual's need for physical touch. Findings indicate that the need for touch does not explain the variance between individuals. We conclude that touch screen technology complements the physical retail environment.     

Comparison of Cognitive Modeling and User Performance Analysis for Touch Screen Mobile Interface Design

The aim of this study is to analyze and comparatively evaluate the usability of touch screen mobile applications through cognitive modeling and end-user usability testing methodologies. The study investigates the accuracy of the estimated results of a cognitive model produced for touch screen mobile phone interfaces. A mobile wallet application was chosen as the mobile software. The CogTool modeling tool was used as the cognitive modeling method. Eight tasks were determined and user tests were conducted in a usability laboratory with 10 participants. The tasks were compared on the basis of step time and total task completion time. This study reveals that CogTool gives approximate estimations with actual user performance on touch screen mobile phone application interfaces. However, if there are special cases in the tasks such that users are very accustomed to the steps or decision-making that is involved in the tasks, the “Think Operation” in CogTool should be modified.     

Enhanced auditory feedback for Korean touch screen keyboards

With the increasing popularity of touch screen mobile devices, improving the usability and the user experience while inputting text on these devices is becoming increasingly important. Most conventional touch screen keyboards on mobile devices rely heavily on visual feedback, while auditory feedback seldom includes any useful information about what is being inputted by the user. The auditory feedback usually simply replicates the sounds produced by a physical keyboard. This paper describes the development of an enhanced auditory feedback mechanism for a Korean touch screen keyboard called the enhanced auditory feedback (EAF) mechanism. EAF has subtle phonetic auditory feedback generated using the acoustic phonetic features of human speech. While typing with EAF, users can acquire non-invasive auditory clues about the keys pressed. In this work, we compare conventional auditory feedback for a touch screen keyboard used in touch screen mobile devices with that of EAF and explore the possibility of using enhanced auditory feedback for touch screen keyboards.     

Improving the Performance of Input Interfaces Through Scaling and Human Motor Models

The performance of interfaces is affected by human factors, which vary from one person to another, and by the inherent characteristics of the various devices involved. A set of techniques has been studied in order to improve the efficiency and efficacy of input interface devices. These techniques are based on the modification of the motor scaling factor, a transformation similar to the known Control-Display ratio (CD ratio). Operation time, the accuracy of the task and user workload are the indicators used in this work. By means of models based on the various human motor behaviors, the improvement of such indicators has been demonstrated. Using some common input interface devices, a number of experiments have been carried out to evaluate the presented methodology. The results show that the overall performance of input interfaces is significantly improved by applying such methodology.     

The impact of on-road motion on BMS touch screen device operation

This study investigates the effect of vehicle motion on performance, usability and workload for a touch screen in-vehicle Battle Management System (BMS). Participants performed a series of battle management tasks while a vehicle was driven over sealed (characteristic of 'normal' vehicle motion) and unsealed (characteristic of 'high' vehicle motion) roads. The results indicate that unsealed road conditions impair the performance of information input tasks (tasks that require the user to enter information, e.g. text entry) but not information extraction tasks (tasks that require the user to retrieve information from the system, e.g. reading coordinates). Participants rated workload as higher and the system as less usable on the unsealed road. In closing, the implications for in-vehicle touch screen design and use in both military and civilian driving contexts are discussed. Practitioner Summary: The effect of motion on interacting with in-vehicle touch screen devices remains largely unexplored. This study examines the effect of different levels of vehicle motion on the use of a BMS. Using the system under off-road conditions had a detrimental impact on workload, performance and usability.     

Electronic sketching on a multi-platform context: A pilot study with developers

During the past 45 years there has been a recurrence of interest on supporting sketching at electronic devices and interactive surfaces, and despite being sketching recognition fairly well addressed on the literature, the adoption of electronic sketching as a design tool is still a challenge.The current popularization of touch screen devices allows designers to sketch using their device of preference, while the current multi-platform capabilities made possible by HTML5 allows sketching systems to run on many devices at the same time. Those two factors combined might pose new opportunities for researchers to explore how designers use sketching on flexible setups by combining heterogeneous sketching devices for design sessions.This may arise new possibilities in the field of prototyping user interfaces since, by using such multi-platform systems, designers would now be able of designing interfaces for multiple devices by producing and testing them on the device itself.This paper reports a pilot experiment conducted with 6 developers, grouped into pairs on design sessions using Gambit – a multi-platform sketching system that provides a lightweight approach for prototyping user interfaces for many devices at once. We performed a discourse analysis of the professionals based on recorded videos of interviews conducted during and after design sessions with the system and aggregated the data in order to investigate the main requirements for multi-platform sketching systems.     

Advances in interactive display technologies

How we interface and interact with computing, communications, and entertainment devices is going through revolutionary changes, with natural user inputs based on touch, gesture, and voice replacing or augmenting the use of traditional user interfaces based on the keyboard, mouse, trackballs, joysticks, and so forth. In particular, mobile devices have rapidly transitioned to adopt touch screen technologies as the primary means of user interface during recent years, enabling fun new interactive applications and experiences for the users. We have dedicated this special section of the Journal of the Society for Information Display to present an overview of recent developments and trends in the emerging field of interactive display technologies.     

Linux平台下支持触摸屏的Qt／Embedded移植

随着嵌入式的发展,触摸屏以友善的人机交互性、操作简单灵活、输入速度快,已逐渐取代键盘,成为嵌入式系统的主流输入设备;而Qt作为跨平台的图形用户界面工具包诞生以后,已经扩展到了包括便携式设备在内的几乎当今程序设计的所有领域。以S3C2416为例,介绍了在Linux平台下支持触摸屏的Qt／Embedded移植的详细过程,包括系统环境介绍、带触摸屏库的Qt／Embedded的编译、Linux文件系统设置等,并挑选了一个demo程序对移植情况进行了检验。最后对支持触摸屏的嵌入式系统作了初步展望。     

Forward/up directional incompatibilities during cursor placement within graphical user interfaces

Within graphical user interfaces, an indirect relationship between display and control may lead to directional incompatibilities when a forward mouse movement codes upward cursor motions. However, this should not occur for left/right movements or direct cursor controllers (e.g. touch sensitive screens). In a four-choice reaction time task, 12 participants performed movements from a central start location to a target situated at one of four cardinal points (top, bottom, left, right). A 2 × 2 × 2 design varied directness of controller (moving cursor on computer screen or pen on graphics tablet), compatibility of orientation of cursor controller with screen (horizontal or vertical) and axis of desired cursor motion (left/right or up/down). Incompatibility between orientation of controller and motion of cursor did not affect response latencies, possibly because both forward and upward movements are away from the midline and go up the visual field. However, directional incompatibilities between display and controller led to slower movement with prolonged accelerative phases. Indirect relationships between display and control led to less efficient movements with prolonged decelerative phases and a tendency to undershoot movements along the bottom/top axis. More direct cursor control devices, such as touch sensitive screens, should enhance the efficiency of aspects of cursor trajectories.     

Biomechanical analysis of upper limb during the use of touch screen: motion strategies identification

Abstract

Nowadays touch technology is growing and developers try to make it ever more intuitive and easier to use. This present work focused on the upper limb joint coordination during the achievement of puzzles on touch screen. A 5-inch and 10-inch devices were used to perform 9 and 16 pieces puzzles dragged with digits. The conclusions showed an increase in joint solicitation with the number of piece and the touch screen size. Moreover, three interactions strategies proved to be an evidence: the ‘wrist strategy’ preferentially implying wrist flexion/extension, the ‘elbow strategy’ preferentially implying the elbow flexion/extension and the ‘neutral strategy’ mobilising equally the two joints. From an ergonomic point of view, the data about how the upper limb segments are mobilised while interacting with the screen could be relevant to increase the adaptability of the devices to the user, including users with motor impairments.

Practitioner Summary: Information about the biomechanical organisation of movement during interaction with touch devices appears relevant in order to develop applications adapted to the motor capacities of users. From the analysis of joint angles when performing several times a puzzle with healthy subjects, three motor strategies were highlighted.     

Forward/up directional incompatibilities during cursor placement within graphical user interfaces

Within graphical user interfaces, an indirect relationship between display and control may lead to directional incompatibilities when a forward mouse movement codes upward cursor motions. However, this should not occur for left/right movements or direct cursor controllers (e.g. touch sensitive screens). In a four-choice reaction time task, 12 participants performed movements from a central start location to a target situated at one of four cardinal points (top, bottom, left, right). A 2 x 2 x 2 design varied directness of controller (moving cursor on computer screen or pen on graphics tablet), compatibility of orientation of cursor controller with screen (horizontal or vertical) and axis of desired cursor motion (left/right or up/down). Incompatibility between orientation of controller and motion of cursor did not affect response latencies, possibly because both forward and upward movements are away from the midline and go up the visual field. However, directional incompatibilities between display and controller led to slower movement with prolonged accelerative phases. Indirect relationships between display and control led to less efficient movements with prolonged decelerative phases and a tendency to undershoot movements along the bottom/top axis. More direct cursor control devices, such as touch sensitive screens, should enhance the efficiency of aspects of cursor trajectories.