Looks are (almost) everything: where drivers look to get information

OBJECTIVE: To describe the impact of Rockwell's early eye movements research. BACKGROUND: The advent of a new technology enabling measurements of eye movements in natural environments launched the seminal research of a Human Factors pioneer, Tom Rockwell, into how drivers process visual information. METHOD: In two seminal Human Factors articles -"Mapping Eye-Movement Pattern to the Visual Scene in Driving: An Exploratory Study" (Mourant & Rockwell, 1970) and "Strategies of Visual Search by Novice and Experienced Drivers" (Mourant & Rockwell, 1972)- Rockwell and his student, Ron Mourant, examined drivers' eye movements in naturalistic driving environments. RESULTS: The analyses of the visual fixations revealed systematic relationships between the sources of information the drivers needed to drive safely and the spatial distributions of their visual fixations. In addition, they showed that as drivers gain skill and experience, their pattern of fixations changes in a systematic manner. CONCLUSIONS: The research demonstrated that fixations and saccadic eye movements provide important insights into drivers' visual search behavior, information needs, and information acquisition processes. APPLICATION: This research has been a cornerstone for a myriad of driving-related studies, by Rockwell and other researchers. Building on Rockwell's pioneering work, these studies used eye-tracking systems to describe cognitive aspects of skill acquisition, and the effects of fatigue and other impairments on the process of attention and information gathering. A novel and potentially revolutionary application of this research is to use eye movement recordings for vehicle control and activation of in-vehicle safety systems.     

Parallel extremal optimization in processor load balancing for distributed applications

The paper concerns parallel methods for extremal optimization (EO) applied in processor load balancing in execution of distributed programs. In these methods EO algorithms detect an optimized strategy of tasks migration leading to reduction of program execution time. We use an improved EO algorithm with guided state changes (EO-GS) that provides parallel search for next solution state during solution improvement based on some knowledge of the problem. The search is based on two-step stochastic selection using two fitness functions which account for computation and communication assessment of migration targets. Based on the improved EO-GS approach we propose and evaluate several versions of the parallelization methods of EO algorithms in the context of processor load balancing. Some of them use the crossover operation known in genetic algorithms. The quality of the proposed algorithms is evaluated by experiments with simulated load balancing in execution of distributed programs represented as macro data flow graphs. Load balancing based on so parallelized improved EO provides better convergence of the algorithm, smaller number of task migrations to be done and reduced execution time of applications.     

Influence of social presence on eye movements in visual search tasks

Abstract

This study employed an eye-tracking technique to investigate the influence of social presence on eye movements in visual search tasks. A total of 20 male subjects performed visual search tasks in a 2 (target presence: present vs. absent) × 2 (task complexity: complex vs. simple) × 2 (social presence: alone vs. a human audience) within-subject experiment. Results indicated that the presence of an audience could evoke a social facilitation effect on response time in visual search tasks. Compared with working alone, the participants made fewer and shorter fixations, larger saccades and shorter scan path in simple search tasks and more and longer fixations, smaller saccades and longer scan path in complex search tasks when working with an audience. The saccade velocity and pupil diameter in the audience-present condition were larger than those in the working-alone condition. No significant change in target fixation number was observed between two social presence conditions.

Practitioner Summary: This study employed an eye-tracking technique to examine the influence of social presence on eye movements in visual search tasks. Results clarified the variation mechanism and characteristics of oculomotor scanning induced by social presence in visual search.     

Focusing in algorithm explanation

Algorithm animation attempts to explain an algorithm by visualizing interesting events of the execution of the implemented algorithm on some sample input. Algorithm explanation describes the algorithm on some adequate level of abstraction, states invariants, explains how important steps of the algorithm preserve the invariants, and abstracts from the input data up to the relevant properties. It uses a small focus onto the execution state. This paper is concerned with the explanation of algorithms on linked data structures. The thesis of the paper is that shape analysis of such algorithms produces abstract representations of such data structures, which focus on the “active” parts, i.e., the parts of the data structures, which the algorithm can access during it's next steps. The paper presents a concept of visually executing an algorithm on these abstract representations of data     

Automated software test data generation

An alternative approach to test-data generation based on actual execution of the program under test, function-minimization methods and dynamic data-flow analysis is presented. Test data are developed for the program using actual values of input variables. When the program is executed, the program execution flow is monitored. If during program execution an undesirable execution flow is observed then function-minimization search algorithms are used to automatically locate the values of input variables for which the selected path is traversed. In addition, dynamic data-flow analysis is used to determine those input variables responsible for the undesirable program behavior, significantly increasing the speed of the search process. The approach to generating test data is then extended to programs with dynamic data structures and a search method based on dynamic data-flow analysis and backtracking is presented. In the approach described, values of array indexes and pointers are known at each step of program execution; this information is used to overcome difficulties of array and pointer handling     

Display Movement Velocity and Dynamic Visual Search Performance

In practice, many visual search tasks are performed under dynamic conditions. An experiment was conducted here to test visual search strategy adopted by a person in a dynamic visual search and to investigate the effects of display movement velocity on search time and detection accuracy in it. Thirty‐five participants were randomly tested with all 10 angular velocities of 0, 2, 4, 6, 8, 10, 12, 14, 16, and 32 deg/s. The data obtained fitted the random search model well. The results revealed that observers utilized a random search strategy during the dynamic visual search process and that display movement velocity influenced search performance. In comparison with static visual search, an angular velocity faster than 4 deg/s resulted in a significant decrement in search performance. The variations of duration of individual fixations, the probability of target detection in a single fixation and visual lobe area were discussed. The obtained relationships between display movement velocity, search time and detection accuracy can serve as a useful guide for designing a dynamic search task, thus helping to maximize the cost‐effectiveness of dynamic search tasks while minimizing errors and misses during the search process.     

Dynamic method for software test data generation

Test data generation in program testing is the process of identifying a set of test data which satisfies a given testing criterion. Existing pathwise test data generators proceed by selecting program paths that satisfy the selected criterion and then generating program inputs for these paths. One of the problems with this approach is that unfeasible paths are often selected; as a result, significant computational effort can be wasted in analysing those paths. In this paper, an approach to test data generation, referred to as a dynamic approach for test data generation, is presented. In this approach, the path selection stage is eliminated. Test data are derived based on the actual execution of the program under test and function minimization methods. The approach starts by executing a program for an arbitrary program input. During program execution for each executed branch, a search procedure decides whether the execution should continue through the current branch or an alternative branch should be taken. If an undesirable execution flow is observed at the current branch, then a real-valued function is associated with this branch, and function minimization search algorithms are used to locate values of input variables automatically, which will change the flow of execution at this branch.     

A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction

Human visual search plays an important role in many human–computer interaction (HCI) tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as “active vision” (Findlay & Gilchrist, 2003). The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks—and at least partially answers—the four questions of active vision: (a) What can be perceived in a fixation? (b) When do the eyes move? (c) Where do the eyes move? (d) What information is integrated between eye movements? Answers include: (a) Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. (b) The eyes move after the fixated visual stimulus has been processed (i.e., has entered working memory). (c) The eyes tend to go to nearby objects. (d) Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives HCI researchers and practitioners a better understanding of how people visually interact with computers, and provides a theoretical foundation for predictive analysis tools that can predict aspects of that interaction.     

Derivation and test of an optimum overlapping-lobes model of visualsearch

The visual search process is modeled as a sequence of fixations which may or may not partially overlap in coverage of the search field. A single expression is derived for the probability of search success which covers all overlap cases. Making the further assumption that search strategy is chosen so as to maximize expected value, yields the result that each point on the search field must be fixated an integral number of times. In an experiment where visual lobe size and fixation duration were controlled, human subjects behaved in a manner close to that predicted by the optimization model. It is concluded that further development of the model using less restrictive assumptions is warranted     

Perceived Visual Complexity of In-Vehicle Information Display and Its Effects on Glance Behavior and Preferences

Despite enhancements in the visual complexity of in-vehicle information display in recent years, few studies have examined the effects of such increased complexity. We conducted this study with the following objectives: (1) to suggest a framework for predicting the perceived visual complexity (PVC) of in-vehicle information display; (2) to examine the effects of PVC on the visual behavior of human operators; (3) to investigate the relationship between preferences and PVC. A theoretical framework to evaluate PVC was developed, and a survey study was used to collect participants’ perceptions on visual complexity. A regression analysis was employed to find the relationship between each of three factors and PVC. Two of the factors—quantity and variety—showed a positive correlation with PVC, whereas the third factor, relation, exhibited a negative correlation. Visual search experiments were conducted to test the effects of PVC on the performance of visual search tasks and glance behavior. The results showed that the high level of PVC leads to more time-on-task and number of fixations. We also found that preference for in-vehicle information displays was inversely proportional to PVC. The results enable us to predict how human operators perceive visual complexity and explain the influence of PVC on human behavior.     

试论汉字数字输入法评价

GB18031对如何科学评价数字输入法起到非常重要的指导作用。但是GB18031的部分性能指标在执行时存在一定的难度。数字输入法软件功能至今缺乏相应的国家标准。建立科学的数字输入法软件功能国家标准成为相当迫切的问题。本文讨论了GB18031中规定的易学性的可判定性,并对重码键选率的执行难点进行了定量分析。本文还分析了面向普及型汉字录入人员的数字输入法的特点,对进一步完善GB18031提出了具体的建议。本文还说明了建立数字输入法功能国家标准的必要性,对如何建立数字输入法功能国家标准进行了初步研究。     

Optimizing top-k selection queries over multimedia repositories

Repositories of multimedia objects having multiple types of attributes (e.g., image, text) are becoming increasingly common. A query on these attributes will typically, request not just a set of objects, as in the traditional relational query model (filtering), but also a grade of match associated with each object, which indicates how well the object matches the selection condition (ranking). Furthermore, unlike in the relational model, users may just want the k top-ranked objects for their selection queries for a relatively small k. In addition to the differences in the query model, another peculiarity of multimedia repositories is that they may allow access to the attributes of each object only through indexes. We investigate how to optimize the processing of top-k selection queries over multimedia repositories. The access characteristics of the repositories and the above query model lead to novel issues in query optimization. In particular, the choice of the indexes used to search the repository strongly influences the cost of processing the filtering condition. We define an execution space that is search-minimal, i.e., the set of indexes searched is minimal. Although the general problem of picking an optimal plan in the search-minimal execution space is NP-hard, we present an efficient algorithm that solves the problem optimally with respect to our cost model and execution space when the predicates in the query are independent. We also show that the problem of optimizing top-k selection queries can be viewed, in many cases, as that of evaluating more traditional selection conditions. Thus, both problems can be viewed together as an extended filtering problem to which techniques of query processing and optimization may be adapted.     

On the advantages of foveal mechanisms for active stereo systems in visual search tasks

In this work we study how information provided by foveated images sampled according to the log-polar transformation can be integrated over time in order to build accurate world representations and accomplish visual search tasks in an efficient manner. We focus on a specific visual information modality depth and on how to store it in a flexible memory structure. We propose a probabilistic observational model for a stereo system that relies on the Unscented Transform in order to propagate uncertainty in stereo matching, due to spatial quantization in the retina, to the 3D Cartesian domain. Probabilistic depth measurements are integrated in a novel Sensory Ego-Sphere whose topology can be biased with foveal-like distributions, according to the autonomous agent short-term tasks and goals. Furthermore, we investigate an Upper Confidence Bound algorithm for the task of simultaneously finding the closest object to the observer (visual search) and learning the surrounding environment 3D map (mapping). The performance of task execution is assessed both with a foveated log-polar sensor and a classical uniform one. The advantage of foveal vision and custom ego-sphere representations are illustrated in a series of experiments with a realistic simulator.     

Conspicuity in realistic scenes: an eye-movement measure

An experiment is described to measure the distracting effects of advertisements on the conspicuity of routing signs in realistic scenes. Slides of railway station scenes were shown in which subjects had to search for a target word used in a routing sign present in the scene. Eye movements were recorded to determine search time and number of fixations during search time. Both search time and number of fixations increased systematically with the number of advertisements in two of the three experimental scenes. The distribution of fixations over the scenes is discussed.     

Visual understanding industrial workflows under uncertainty on distributed service oriented architectures

Service Oriented Architecture (SOA) is an evolution of distributed computing based on the request/reply design paradigm for synchronous and asynchronous applications. In such architectures, application developers or system integrators can build scenarios by composing one or more services without knowing the services’ underlying implementations. In this paper, we adopt a SOA framework for on-line scheduling complex industrial workflows. The proposed SOA consists of three main layers; the input layer, the benchmarking layer, the prediction layer and the scheduler. The main innovation of our SOA architecture is the prediction layer that supports computer vision tools able to visually observe how an industrial workflow is actually executed. This way, we can automatically estimate the actual execution time for a process. In particular a new self-initialized visual tracker algorithm is proposed in this paper to robustly trace workers’ trajectory in a plant via visual observations. Then, part-to-whole curve matching is presented so as to find correspondences among the traced curve and the ideal one and thus improving scheduler efficiency. The input layer interoperably describes industrial operations using the XPDL (an XML-based) format. The benchmarker evaluates much faster than real-time and in an off-line mode how long it takes for an industrial workflow to be executed on a given resource guaranteeing an almost real-time implementation of the video processing algorithms on the plant. The last component of the proposed SOA is the scheduler with the goal to assign the workflows to the available resources. In this paper, a Maximum Benefit First (MBF) scheduler is presented which maximizes the total gain received by the industry when completing execution of all the beneficial operations with minimum violations of their delivery deadlines (that is with the minimum compensations). Experiments have been conducted on a real-world industrial plant of Nissan Iberica automobile construction indicating the efficiency of the proposed system.     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

Eye movements of younger and older drivers.

This 2-part study focuses on eye movements to explain driving-related visual performance in younger and older persons. In the first task, participants' eye movements were monitored as they viewed a traffic scene image with a numeric overlay and visually located the numbers in their sequential order. The results showed that older participants had significantly longer search episodes than younger participants, and that the visual search of older adults was characterized by more fixations and shorter saccades, although the average fixation durations remained the same. In the second task, participants viewed pictures of traffic scenes photographed from the driver's perspective. Their task was to assume the role of the driver and regard the image accordingly. Results in the second task showed that older participants allocated a larger percentage of their visual scan time to a small subset of areas in the image, whereas younger participants scanned the images more evenly. Also, older participants revisited the same areas and younger participants did not. The results suggest how aging might affect the efficacy of visual information processing. Potential applications of this research include training older drivers for a more effective visual search, and providing older drivers with redundant information in case some information is missed.     

ADAPT: A Predictive Cognitive Model of User Visual Attention and Action Planning

We present a computational cognitive model of novice and expert aviation pilot action planning called ADAPT that models performance in a dynamically changing simulated flight environment. We perform rigorous tests of ADAPT's predictive validity by comparing the performance of individual human pilots to that of their respective models. Individual pilots were asked to execute a series of flight maneuvers using a flight simulator, and their eye fixations and control movements were recorded in a time-synched database. Computational models of each of the 25 individual pilots were constructed, and the individual models simulated execution of the same flight maneuvers performed by the human pilots. The time-synched eye fixations and control movements of individual pilots and their respective models were compared, and rigorous tests of ADAPT's predictive validity were performed. The model explains and predicts a significant portion of pilot visual attention and control movements during flight as a function of piloting expertise. Implications for adaptive training systems are discussed.     

Predicting user attention in complex web pages

Users' visual attention measured by eyetracking fixations was investigated in web pages with different designs. Browsing and search conditions were tested. Layout structure influenced attention with fixation densities being concentrated in upper parts of pages according to the layout structure. In sites with open graphical layouts, animations and images dominated attention. In the search condition, attention patterns focused on salient objects and information-scent components leading to the targets. Based on the results, a model of structured directed visual attention was proposed and implemented in the Web Page Analyser tool (WPA) to predict heat maps of visual attention. Validation of the tool demonstrated good accuracy in browse and search modes.     

A two visual systems approach to understanding voice and gestural interaction

It is important to consider the physiological and behavioral mechanisms that allow users to physically interact with virtual environments. Inspired by a neuroanatomical model of perception and action known as the two visual systems hypothesis, we conducted a study with two controlled experiments to compare four different kinds of spatial interaction: (1) voice-based input, (2) pointing with a visual cursor, (3) pointing without a visual cursor, and (4) pointing with a time-lagged visual cursor. Consistent with the two visual systems hypothesis, we found that voice-based input and pointing with a cursor were less robust to a display illusion known as the induced Roelofs effect than pointing without a cursor or even pointing with a lagged cursor. The implications of these findings are discussed, with an emphasis on how the two visual systems model can be used to understand the basis for voice and gestural interactions that support spatial target selection in large screen and immersive environments.