A switched dynamic model for pointing tasks with a computer mouse

In this paper, we study one of the most fundamental tasks in human‐computer interaction, the pointing task. It can be described simply as reaching a target with a cursor starting from an initial position (e.g. executing a movement using a computer mouse to select an icon). In this paper, a switched dynamic model is proposed to handle cursor movements in indirect pointing tasks. The model contains a ballistic movement phase governed by a nonlinear model in Lurie form and a corrective movement phase described by a linear visual‐feedback system. The stability of the model is first established and the derived model is then validated with experimental data acquired in a pointing task with a mouse. It is established that the measured data of pointing movements of different types can be fitted within the proposed model. Numerical comparison against pointing models available in the literature is also provided.     

Dynamic cursor gain and tactual feedback in the capture of cursor movements

Recent research involving a trackball with force feedback has demonstrated that tactile feedback can enhance the acquisition of targets in graphical user interfaces in terms of movement times and errors. The present study seeks to explore the degree to which tactual feedback over a target, in contrast to changes in the display/control gain over the target, influences target acquisition performance. Tactual feedback over a target is felt as a pulling force towards the centre of a target, with a counterforce applied when moving out of the centre. Changes in the cursor gain can be used to create a cursor-catching effect by requiring more movement effort of the control device to leave than to enter the target centre, without increasing the total amount of effort to enter and leave the target area. User movement in entering a target is thus braked by the change in cursor gain. Results of an experiment indicated that target acquisition performance was generally higher in the tactual feedback condition, followed by cursor gain feedback, in comparison with no-cursor gain feedback. User interface design issues as related to gain feedback in visual interfaces and tactual feedback over targets are considered.     

Adjustment to a complex visuo-motor transformation at early and late working age

Age-related changes of adjustment to visuo-motor transformations were studied for a complex transformation modelled after those encountered in laparoscopic surgery. Movement times of aimed movements were initially almost identical for the two age groups and diverged in the course of practice. In test phases without visual feedback, no age-related variation of the adaptive shifts and after effects of amplitude and curvature of hand movements were observed, but only of their direction. Directional adaptive shifts were bimodally distributed, with one mode near to perfect adaptation (‘adapters’) and the other one near to absence of adaptation (‘non-adapters’). Among the young participants, adapters were more frequent than among the old participants. These findings extend previous results on age-related changes of adjustment to simple transformations to complex transformations. They are consistent with the claim that age-related changes of adjustment to visuo-motor rotations come about primarily by impairments of strategic corrections. A link is established between basic-research findings and transformations encountered outside the laboratory: what suffers at higher working age is explicit knowledge of visuo-motor rotations and the strategic corrections based on it. This suggests the provision of opportunities for explicit-knowledge acquisition for those of a higher age.     

Reconstruction of human neuromuscular control signals using a sliding mode control technique

A sliding mode controller, developed using MATLAB/SIMULINK, is incorporated within an existing model of the complex nonlinear system of the human leg with the aim of identifying the operation of the feedback control system within the body to produce the observed leg movement during walking. The control design procedure first defines a switching function, the so-called sliding surface, which prescribes the required dynamic behaviour of the error between the measured data and the model output. A discontinuous feedback control law is then designed to drive the system trajectories onto the sliding surface and force them to remain there. The attractiveness of sliding mode control for the analysis of biomechanical control systems arises from the equivalent control principle. If a switching surface is selected which ensures exact tracking between the model trajectory and experimentally measured trajectory from a human subject, then the filtered discontinuous control signals from the simulation model provide estimates of the signals required to achieve the observed movement. The inherently robust properties of sliding mode control ensure precise tracking by the model of the actual leg movement, thus enabling the feedback control signal required to carry out the movement to be reconstructed.     

Testing usability and trainability of indirect touch interaction: perspective for the next generation of air traffic control systems

This study aims to determine whether indirect touch device can be used to interact with graphical objects displayed on another screen in an air traffic control (ATC) context. The introduction of such a device likely requires an adaptation of the sensory-motor system. The operator has to simultaneously perform movements on the horizontal plane while assessing them on the vertical plane. Thirty-six right-handed participants performed movement training with either constant or variable practice and with or without visual feedback of the displacement of their actions. Participants then performed a test phase without visual feedback. Performance improved in both practice conditions, but accuracy was higher with visual feedback. During the test phase, movement time was longer for those who had practiced with feedback, suggesting an element of dependency. However, this ‘cost’ of feedback did not extend to movement accuracy. Finally, participants who had received variable training performed better in the test phase, but accuracy was still unsatisfactory. We conclude that continuous visual feedback on the stylus position is necessary if tablets are to be introduced in ATC.     

Frame of reference and adaptation to directional bias in a video-controlled reaching task

The present study (N=56) investigated spatio-temporal accuracy of horizontal reaching movements controlled visually through a vertical video monitor. Direct vision of the hand was precluded and the direction of hand trajectory, as perceived on the video screen, was varied by changing the angle of the camera. The orientation of the visual scene displayed on the fronto-parallel plane was thus congruent (0 degrees condition) or non-congruent (directional bias of 15 degrees, 30 degrees or 45 degrees counterclockwise) according to the horizontal working space. The goal of this study was to determine whether local learning of a directional bias can be transferred to other locations in the working space, but taking into account the magnitude of the directional bias (15 degrees, 30 degrees or 45 degrees ), and the position of the successive objectives (targets at different distances (TDD) or different azimuths (TDA)). Analysis of the spatial accuracy of pointing movements showed that when introducing a directional bias, terminal angular error was linearly related to the amount of angular perturbation (around 30%). Seven trials were, on average, necessary to eliminate this terminal error, whatever the magnitude of the directional bias and the position of the successive targets. When changing the location of the spatial objective, transfer of adaptation was achieved in the TDD condition but remained partial in the TDA condition. Furthermore, initial orientation of the trajectory suggested that some participants used a hand-centred frame of reference whereas others used an external one to specify movement vector. The adaptation process differed as a function of the frame of reference used, but only in the TDA condition. Adaptation for participants using a hand-centred frame of reference was more concerned with changes in the shape of the trajectory, whereas participants using an external frame of reference adapted their movement by up-dating the initial direction of hand trajectory. As a whole, these findings suggest that the processes involved in remote visual control of hand movement are complex with the result that tasks requiring video-controlled manipulation like video-controlled surgery require specific spatial abilities in actors and consequential plasticity of their visuo-motor system, in particular concerning the selection of the frame of reference for action.     

Frame of reference and adaptation to directional bias in a video-controlled reaching task

The present study (N=56) investigated spatio-temporal accuracy of horizontal reaching movements controlled visually through a vertical video monitor. Direct vision of the hand was precluded and the direction of hand trajectory, as perceived on the video screen, was varied by changing the angle of the camera. The orientation of the visual scene displayed on the fronto-parallel plane was thus congruent (0° condition) or non-congruent (directional bias of 15°, 30° or 45° counterclockwise) according to the horizontal working space. The goal of this study was to determine whether local learning of a directional bias can be transferred to other locations in the working space, but taking into account the magnitude of the directional bias (15°, 30° or 45°), and the position of the successive objectives (targets at different distances (TDD) or different azimuths (TDA)). Analysis of the spatial accuracy of pointing movements showed that when introducing a directional bias, terminal angular error was linearly related to the amount of angular perturbation (around 30%). Seven trials were, on average, necessary to eliminate this terminal error, whatever the magnitude of the directional bias and the position of the successive targets. When changing the location of the spatial objective, transfer of adaptation was achieved in the TDD condition but remained partial in the TDA condition. Furthermore, initial orientation of the trajectory suggested that some participants used a hand-centred frame of reference whereas others used an external one to specify movement vector. The adaptation process differed as a function of the frame of reference used, but only in the TDA condition. Adaptation for participants using a hand-centred frame of reference was more concerned with changes in the shape of the trajectory, whereas participants using an external frame of reference adapted their movement by up-dating the initial direction of hand trajectory. As a whole, these findings suggest that the processes involved in remote visual control of hand movement are complex with the result that tasks requiring video-controlled manipulation like video-controlled surgery require specific spatial abilities in actors and consequential plasticity of their visuo-motor system, in particular concerning the selection of the frame of reference for action.     

Cursor type and response conflict in graphical user interfaces

Arrowhead cursor shape may offer irrelevant cues that conflict with desired positioning movements. To assist cursor design this study considered how cursor shape can influence the preparation or execution of cursor positioning movements. An experiment varied cursor shape on-line such that its shape cued the required direction of movement or better afforded hitting the target. Twelve participants performed cursor positioning movements with systematic variations in cursor shape affecting preparation (neutral, pre-cuing target direction), cursor flight (arrowhead, comet) and terminal guidance (big, small target). Kinematic analysis determined the effects on phases of cursor placement. Cursor shape primarily affected terminal guidance and implies conventional cursor designs should be reconsidered.     

Head-operated computer controls: effect of control method on performance for subjects with and without disability

Head-operated computer controls provide an alternative means of computer access for people with physical disabilities. A person's ability to use such head controls may be reduced if he or she experiences neck movement limitations. Five experimental methods of compensating for neck movement limitations were evaluated in comparison to a standard head control interface. Twenty-two subjects without disabilities and three subjects with multiple sclerosis performed icon acquisition exercises using the standard interface and each of the five experimental compensation methods. Subjects without disabilities had less tendency to overshoot the target icons when using an interface with decreased sensitivity or one in which head movements controlled cursor velocity rather than cursor position (p<0.05). Subjects with multiple sclerosis tended to be more accurate when using an interface with increased sensitivity, and had less tendency to overshoot icons when using head movements to control cursor velocity rather than cursor position. Overall, subjects tended to demonstrate faster performance when using an interface with reduced sensitivity.     

移动机器人轨迹跟踪快速终端滑模自抗扰控制

针对移动机器人动力学模型难以精确建立、运动过程中各种干扰对高精度轨迹跟踪造成偏航等问题,构造出一种快速终端滑模自抗扰控制器,实现了高速高精度轨迹跟踪控制目标。首先建立非完整移动机器人的干扰控制模型;然后运用扩张状态观测器实时监测系统未建模动态与各种干扰;同时将扩张状态量和系统反馈量作为快速终端滑模算法的系统变量;最后设计出一种快速终端滑模控制律,代替传统自抗扰算法中的非线性控制律,从而实现位姿的快速精确跟踪。该算法既克服了非线性误差控制律中参数繁杂不易整定的缺陷,又增强了系统鲁棒性和轨迹跟踪的动态品质。对比仿真实验验证了该控制算法的有效性和先进性。     

Improved efficiency through I- and E-feedback: a trackball with contextual force feedback

It has been argued by Engel and Haakma (1993, Expectations and feedback in user-system communication, International Journal of Man-Machine Studies, 39, 427-452) that for user-system communication to become more efficient, machine interfaces should present both early layered I-feedback on the current partial message interpretation as well as layered expectations (E-feedback) concerning the message components still to be communicated. As a clear example of our claim, this paper describes an experimental trackball device that provides the user with the less common E-feedback in addition to the conventional layered I-feedback in the form of the momentary cursor position on the screen and the kinetic forces from the ball. In particular, the machine expresses its expectation concerning the goal position of the cursor by exerting an extra force to the trackball.Two optical sensors and two servo motors are used in the described trackball device with contextual force feedback. One combination of position sensor and servo motor handles the cursor position and tactile feedback along the x -axis, the other combination controls that along the y-axis. By supplying supportive force feedback as a function of the current display contents and the momentary cursor position, the user's movements are guided towards the cursor target position expected by the machine. The force feedback diminishes the visual processing load of the user and combines increased ease of use with robustness of manipulation.Experiments with a laboratory version of this new device have shown that the force feedback significantly enhances speed and accuracy of pointing and dragging, while the effort needed to master the trackball is minimal compared with that for the conventional trackball without force feedback.     

Adaptation and learning in control of voluntary movement by the central nervous system

In order to control voluntary movements, the central nervous system must solve the following three computational problems at different levels: (1) the determination of a desired trajectory in visual coordinates; (2) the transformation of its coordinates into body coordinates; and (3) the generation of motor command. Concerning these problems, relevant experimental observations obtained in the field of neuroscience are briefly reviewed. On the basis of physiological information and previous models, we propose computational theories and a neural network model which account for these three problems. (1) A minimum torque-change model which predicts a wide range of trajectories in human multi-joint arm movements is proposed as a computational model for trajectory formation. (2) An iterative learning scheme is presented as an algorithm which solves the coordinate transformation and the control problem simultaneously. This algorithm can be regarded as a Newton-like method in function spaces. (3) A neural network model for generation of motor command is proposed. This model contains internal neural models of the motor system and its inverse system. The inverse-dynamics model is acquired by heterosynaptic plasticity using a feedback motor command (torque) as an error signal. The hierarchical arrangement of these neural networks and their global control are discussed. Their applications to robotics are also discussed.     

Forward/up directional incompatibilities during cursor placement within graphical user interfaces

Within graphical user interfaces, an indirect relationship between display and control may lead to directional incompatibilities when a forward mouse movement codes upward cursor motions. However, this should not occur for left/right movements or direct cursor controllers (e.g. touch sensitive screens). In a four-choice reaction time task, 12 participants performed movements from a central start location to a target situated at one of four cardinal points (top, bottom, left, right). A 2 x 2 x 2 design varied directness of controller (moving cursor on computer screen or pen on graphics tablet), compatibility of orientation of cursor controller with screen (horizontal or vertical) and axis of desired cursor motion (left/right or up/down). Incompatibility between orientation of controller and motion of cursor did not affect response latencies, possibly because both forward and upward movements are away from the midline and go up the visual field. However, directional incompatibilities between display and controller led to slower movement with prolonged accelerative phases. Indirect relationships between display and control led to less efficient movements with prolonged decelerative phases and a tendency to undershoot movements along the bottom/top axis. More direct cursor control devices, such as touch sensitive screens, should enhance the efficiency of aspects of cursor trajectories.     

Forward/up directional incompatibilities during cursor placement within graphical user interfaces

Within graphical user interfaces, an indirect relationship between display and control may lead to directional incompatibilities when a forward mouse movement codes upward cursor motions. However, this should not occur for left/right movements or direct cursor controllers (e.g. touch sensitive screens). In a four-choice reaction time task, 12 participants performed movements from a central start location to a target situated at one of four cardinal points (top, bottom, left, right). A 2 × 2 × 2 design varied directness of controller (moving cursor on computer screen or pen on graphics tablet), compatibility of orientation of cursor controller with screen (horizontal or vertical) and axis of desired cursor motion (left/right or up/down). Incompatibility between orientation of controller and motion of cursor did not affect response latencies, possibly because both forward and upward movements are away from the midline and go up the visual field. However, directional incompatibilities between display and controller led to slower movement with prolonged accelerative phases. Indirect relationships between display and control led to less efficient movements with prolonged decelerative phases and a tendency to undershoot movements along the bottom/top axis. More direct cursor control devices, such as touch sensitive screens, should enhance the efficiency of aspects of cursor trajectories.     

A trivial and efficient learning method for motion and force control

The feedback produced by the linear controller of a manipulator executing a trajectory corresponds approximately to the inverse dynamics necessary to drive this trajectory. When a single movement is repeated, the feedback measured in one run can therefore be used as feedforward for the next runs. The learning position–force controller introduced in this paper is based on this idea and a parallel control of force and position. Experiments on 2- and 3-degree-of-freedom parallel manipulators show the simplicity of its implementation and its efficiency for gradually improving trajectory control in repeated movements. The control is robust to high level of noise and the performance is superior than with a parametric controller based on the rigid-body dynamic model. Simulations suggest that these properties also hold when the force is controlled simultaneously to the position.     

Computerized control system and interface for flexible micromanipulator control

Micro- and nanomanipulators are essential for a broad range of applications requiring precise micro- and nanoscopic spatial control such as those in micromanufacturing and single cell analysis. These manipulators are often manually controlled using an attached joystick and can be difficult for operators to use efficiently. This paper describes a system developed in MATLAB to control a well-known, commercial micromanipulator in a user friendly and versatile manner through a graphical user interface (GUI). The control system and interface allows several types of flexible movement controls in three-axis, Cartesian space, including single movements, multiple queued movements, and mouse-following continuous movements. The system uses image processing for closed loop feedback to ensure precise and accurate control over the movement of the manipulator’s end effector. The system can be used on any electronic device capable of running the free MATLAB Runtime Environment (MRE) and the system is extensible to simultaneously control other instruments capable of serial communication.     

What does keeping one's eye on the ball mean?

Abstract

Calling players' attention to keep their eyes on the ball is a constant exhortation in ball sports. Nevertheless, speeds reached by balls are generally superior to the oculo-motor capacity of pursuit. This paradox was investigated by analysing the visuo-motor behaviour of five expert table tennis players using a video-oculographic recording technique. Three different strokes were analysed- forehand (FH), forehand with top spin (TS) and backhand (BH) drives.

The results indicate that it is not necessary to keep one's eye on the ball throughout the entire trajectory. At only two points in the ball's flight was the eye directed towards the ball. These points occurred during the following operations (1) the visual pursuit of the ball at the very beginning of the trajectory and (2) the visuo-motor co-ordination during the execution of the strike. The visual behaviour involved during these phases was analysed in detail. It appeared that visuo-motor behaviour is determined by the external constraints of the stroke (the eccentricity of the ball trajectory in relation to the mid-line of the body and the accuracy of the strike). It appeared that only the first part of the trajectory was visually tracked, beginning just after the opponent's strike. Nevertheless the nature of this operation corresponded to the type of stroke. The ball was visually tracked more often and for a longer period when it moved towards the body (in BH) than when it moved laterally in relation to the body (in FH and TS).

During the strike, the ball was not precisely viewed. A characteristic visuo-motor behaviour was observed. Between the bounce and the strike, the eye was stable inside the orbit and aligned according to the position of the head axis. Both the eye and head together were quickly directed in advance towards the direction of the ball/bat. This mechanism occurred more frequently when the ball was projected laterally in relation to the mid-line (in FH and TS) and mainly when the strike itself was particularly accurate (in TS).

The role of the head and eye orientation towards the target in the execution of fast and accurate table tennis movements is discussed in relation to specific table constraints. Pedagogical consequences of these results are presented.     

Effects of compatibility and turning biases on arrowhead cursor placement in graphical user interfaces

Arrowhead cursors have the potential to generate response conflict, as they are used to signify location but also implicitly cue direction. This study considered the time course of the resolution of any response conflict. The interplay between object properties and action was addressed through consideration of the effects of arrowhead cursor orientation upon action. Twelve participants moved arrowhead cursors of four possible orientations with respect to direction of movement (compatible, incompatible, port, starboard) towards a square (4 mm or 8 mm) target in a graphical user interface. Kinematic analysis determined if and how an interplay existed between object properties and action. Cursor orientation primarily influenced cursor movement, with the effects manifesting during terminal guidance. The blunt end of an arrowhead cursor was moved faster by users, and there appeared to be turning biases during cursor placement. Arrowheads have a number of properties (vertex, orientation, width) that influence cursor placement in graphical user interfaces.     

Endogenous brain–machine interface based on the correlation of EEG maps

In this paper, a non-invasive endogenous brain–machine interface (BMI) based on the correlation of EEG maps has been developed to work in real-time applications. The classifier is able to detect two mental tasks related to motor imagery with good success rates and stability. The BMI has been tested with four able-bodied volunteers. First, the users performed a training with visual feedback to adjust the classifier. Afterwards, the users carried out several trajectories in a visual interface controlling the cursor position with the BMI. In these tests, score and accuracy were measured. The results showed that the participants were able to follow the targets during the performed trajectory, proving that the EEG mapping correlation classifier is ready to work in more complex real-time applications aimed at helping people with a severe disability in their daily life.