Modeling driver behavior in a cognitive architecture

OBJECTIVE: This paper explores the development of a rigorous computational model of driver behavior in a cognitive architecture--a computational framework with underlying psychological theories that incorporate basic properties and limitations of the human system. BACKGROUND: Computational modeling has emerged as a powerful tool for studying the complex task of driving, allowing researchers to simulate driver behavior and explore the parameters and constraints of this behavior. METHOD: An integrated driver model developed in the ACT-R (Adaptive Control of Thought-Rational) cognitive architecture is described that focuses on the component processes of control, monitoring, and decision making in a multilane highway environment. RESULTS: This model accounts for the steering profiles, lateral position profiles, and gaze distributions of human drivers during lane keeping, curve negotiation, and lane changing. CONCLUSION: The model demonstrates how cognitive architectures facilitate understanding of driver behavior in the context of general human abilities and constraints and how the driving domain benefits cognitive architectures by pushing model development toward more complex, realistic tasks. APPLICATION: The model can also serve as a core computational engine for practical applications that predict and recognize driver behavior and distraction.     

Equipping the ACT-R cognitive architecture with a temporal ratio model of memory and using it in a new intelligent adaptive interface

ACT-R, as a useful and well-known cognitive architecture, is a theory for simulating and understanding human cognition. However, the standard version of this architecture uses a deprecated forgetting model. So, we equipped it with a temporal ratio model of memory that has been named as SIMPLE (Scale-Independent Memory, Perception, and Learning). On the other hand, one of the usages of cognitive architectures is to model the user in an Intelligent Adaptive Interface (IAI) implementation. Thus, our motivation for this effort is to use this equipped ACT-R in an IAI to deliver the right information at the right time to users based on their cognitive needs. So, to test our proposed equipped ACT-R, we designed and implemented a new IAI to control a swarm of Unmanned Aerial Vehicles (UAVs). This IAI uses the equipped ACT-R for user cognitive modeling, to deliver the right information to the users based on their forgetting model. Thus, our contributions are: equipping the ACT-R cognitive architecture with the SIMPLE memory model and using this equipped version of ACT-R for user modeling in a new IAI to control a group of UAVs. Simulation results, which have been obtained using different subjective and objective measures, show that we significantly improved situation awareness of the users using the IAI empowered by our equipped ACT-R.

     

Modeling the effect of task and graphical representation on response latency in a graph reading task

We report an investigation into the processes involved in a common graph-reading task using two types of Cartesian graph. We describe an experiment and eye movement study, the results of which show that optimal scan paths assumed in the task analysis approximate the detailed sequences of saccades made by individuals. The research demonstrates the computational inequivalence of two sets of informationally equivalent graphs and illustrates how the computational advantages of a representation outweigh factors such as user unfamiliarity. We describe two models, using the ACT rational perceptual motor (ACT-R/PM) cognitive architecture, that replicate the pattern of observed response latencies and the complex scan paths revealed by the eye movement study. Finally, we outline three guidelines for designers of visual displays: Designers should (a) consider how different quantities are encoded within any chosen representational format, (b) consider the full range of alternative varieties of a given task, and (c) balance the cost of familiarization with the computational advantages of less familiar representations. Actual or potential applications of this research include informing the design and selection of appropriate visual displays and illustrating the practice and utility of task analysis, eye tracking, and cognitive modeling for understanding interactive tasks with external representations.     

ADAPT: A Predictive Cognitive Model of User Visual Attention and Action Planning

We present a computational cognitive model of novice and expert aviation pilot action planning called ADAPT that models performance in a dynamically changing simulated flight environment. We perform rigorous tests of ADAPT's predictive validity by comparing the performance of individual human pilots to that of their respective models. Individual pilots were asked to execute a series of flight maneuvers using a flight simulator, and their eye fixations and control movements were recorded in a time-synched database. Computational models of each of the 25 individual pilots were constructed, and the individual models simulated execution of the same flight maneuvers performed by the human pilots. The time-synched eye fixations and control movements of individual pilots and their respective models were compared, and rigorous tests of ADAPT's predictive validity were performed. The model explains and predicts a significant portion of pilot visual attention and control movements during flight as a function of piloting expertise. Implications for adaptive training systems are discussed.     

Saccade detection during smooth tracking

Saccade detection in an eye-movement trace provides a starting point for analyses ranging from the investigation of low-level oculomotor mechanisms to high-level cognitive processes. When the eye tracks the motion of the object of current interest (smooth pursuit), of the visual background (OKN), or of the resultant visual motion from a head movement (tVOR, rVOR), the smooth tracking movement is generally intermixed with rapid-phase saccadic eye movements, which must be excised to analyze the smooth components of tracking behavior properly. We describe a simple method to detect saccades on a background trace of variable velocity, compare our saccade-detection algorithm with the performance of an expert human observer, and present an ideal-observer analysis to benchmark its detection performance.     

A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction

Human visual search plays an important role in many human–computer interaction (HCI) tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as “active vision” (Findlay & Gilchrist, 2003). The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks—and at least partially answers—the four questions of active vision: (a) What can be perceived in a fixation? (b) When do the eyes move? (c) Where do the eyes move? (d) What information is integrated between eye movements? Answers include: (a) Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. (b) The eyes move after the fixated visual stimulus has been processed (i.e., has entered working memory). (c) The eyes tend to go to nearby objects. (d) Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives HCI researchers and practitioners a better understanding of how people visually interact with computers, and provides a theoretical foundation for predictive analysis tools that can predict aspects of that interaction.     

Utilization of Visual Information Perception Characteristics to Improve Classification Accuracy of Driver’s Visual Search Intention for Intelligent Vehicle

Understanding and predicting a driver’s behaviors in a vehicle is a prospective function embedded in a smart car. Beyond the patterns of observable behaviors, driver’s intention could be identified based on goal-driven behaviors. A computational model to classify driver intention in visual search which is finding a target with one’s eyes as moving selective attention across a search field, could improve the level of intelligence that a smart car could demonstrate. To develop a computational cognitive that explains the underlying cognitive process and reproduces drivers’ behaviors, particular parameters in human cognitive process should be specified. In this study, 2 issues are considered as influential factors on a driver’s eye movements: a driver’s visual information processing characteristics (VIPCs) and the purpose of visual search. To assess an individual’s VIPC, 4 psychological experiments—Donders’s reaction time, mental rotation, signal detection, and Stroop experiments—were utilized. Upon applying k-means clustering method, 114 drivers were divided into 9 driver groups. To investigate the influence of task goal on a driver’s eye movement, driving simulation was conducted to collect a driver’s eye movement data under the given purpose of visual search (perceptual and cognitive tasks). The empirical data showed that there were significant differences in a driver’s oculomotor behavior, such as response time, average fixation time, and average glance duration between the driver groups and the purposes of visual search. The effectiveness of using VIPC for grouping drivers was tested with task goal classification model by comparing the models’ performance when drivers were grouped by typical demographic data such as gender. Results show that grouping based on VIPC improves accuracy and stability of prediction of the model on a driver’s intention underlying visual search behaviors. This study would benefit future studies focusing on personalization and adaptive interfaces in the development of smart car.     

Modeling the Influences of Cyclic Top-Down and Bottom-Up Processes for Reinforcement Learning in Eye Movements

Understanding and reproducing complex human oculomotor behaviors using computational models is a challenging task. In this paper, two studies are presented, which focus on the development and evaluation of a computational model to show the influences of cyclic top-down and bottom-up processes on eye movements. To explain these processes, reinforcement learning was used to control eye movements. The first study showed that, in a picture-viewing task, different policies obtained from different picture-viewing conditions produced different types of eye movement patterns. In another visual search task, the second study illustrated that feedback information from each saccadic eye movement could be used to update the model's eye movement policy, generating different patterns in the following saccade. These two studies demonstrate the value of an integrated reinforcement learning model in explaining both top-down and bottom-up processes of eye movements within one computational model.     

Prediction of human eye movements in facial discrimination tasks

Under natural viewing conditions, human observers selectively allocate their attention to subsets of the visual input. Since overt allocation of attention appears as eye movements, the mechanism of selective attention can be uncovered through computational studies of eyemovement predictions. Since top-down attentional control in a task is expected to modulate eye movements significantly, the models that take a bottom-up approach based on low-level local properties are not expected to suffice for prediction. In this study, we introduce two representative models, apply them to a facial discrimination task with morphed face images, and evaluate their performance by comparing them with the human eye-movement data. The result shows that they are not good at predicting eye movements in this task.     

Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD

This paper presents a real-time framework for computationally tracking objects visually attended by the user while navigating in interactive virtual environments. In addition to the conventional bottom-up (stimulus-driven) saliency map, the proposed framework uses top-down (goal-directed) contexts inferred from the user's spatial and temporal behaviors, and identifies the most plausibly attended objects among candidates in the object saliency map. The computational framework was implemented using GPU, exhibiting high computational performance adequate for interactive virtual environments. A user experiment was also conducted to evaluate the prediction accuracy of the tracking framework by comparing objects regarded as visually attended by the framework to actual human gaze collected with an eye tracker. The results indicated that the accuracy was in the level well supported by the theory of human cognition for visually identifying single and multiple attentive targets, especially owing to the addition of top-down contextual information. Finally, we demonstrate how the visual attention tracking framework can be applied to managing the level of details in virtual environments, without any hardware for head or eye tracking.     

Integrated cognitive architectures: a survey

This article aims to present an account of the state of the art research in the field of integrated cognitive architectures by providing a review of six cognitive architectures, namely Soar, ACT-R, ICARUS, BDI, the subsumption architecture and CLARION. We conduct a detailed functional comparison by looking at a wide range of cognitive components, including perception, memory, goal representation, planning, problem solving, reasoning, learning, and relevance to neurobiology. In addition, we study the range of benchmarks and applications that these architectures have been applied to. Although no single cognitive architecture has provided a full solution with the level of human intelligence, important design principles have emerged, pointing to promising directions towards generic and scalable architectures with close analogy to human brains.     

Eye monitor: microcomputer-based instrument uses an internal modeto track the eye

An eye monitor whose design is inspired by the human visual system is presented. The monitor incorporates an internal representation or model of what the eye looks like to a video camera. The system can measure the position of the eyes and the size of the pupil in the presence of interfering noise and in patients wearing eyeglasses or contact lenses, and it tolerates defocusing due to small movements in depth by the patient. The system makes real-time correction for head and eye movements while measuring pupillary responses to controlled light stimuli. The design and software and hardware components are described, and some applications are noted. Its use for early clinical detection of visual diseases by objectively measuring pupillary responses to carefully controlled light stimuli is examined as an example. Some general observations about using computers in medical measurements are made     

APES: Attentively Perceiving Robot

Robot vision systems—inspired by human-like vision—are required to employ mechanisms similar to those that have proven to be crucial in human visual performance. One of these mechanisms is attentive perception. Findings from vision science research suggest that attentive perception requires a multitude of properties: A retina with fovea-periphery distinction, an attention mechanism that can be manipulated both mechanically and internally, an extensive set of visual primitives that enable different representation modes, an integration mechanism that can infer the appropriate visual information in spite of eye, head, body and target motion, and finally memory for guiding eye movements and modeling the environment. In this paper we present an attentively “perceiving” robot called APES. The novelty of this system stems from the fact that it incorporates all of these properties simultaneously. As is explained, original approaches have to be taken to realize each of the properties so that they can be integrated together in an attentive perception framework.     

基于Navarro示意眼模型的视觉真实感绘制*

结合人眼光学建模和计算机图形学的真实感绘制技术,提出一种基于Navarro示意眼模型的人眼视觉真实感绘制方法.利用Navarro模型与传统的单透镜模型相比能够更精确地模拟人眼的特性,将Navarro模型引入视觉真实感绘制中;采用光线追踪方法,加入非球面折射面的计算,精确地模拟人眼的成像特性.实验结果表明此种方法能够更精...     

Extraction of microsaccade-related signal from single-trial local field potential by ICA with reference

During visual fixation, we unconsciously make tiny, involuntary eye movements or ‘microsaccades’, which have been shown to have a crucial influence on analysis and perception of our visual environment. Given the small size and high irregularity of microsaccades, it is a significant challenge to detect and extract the microsaccade-related neural activities. In this work, we present a novel application of the independent component analysis with reference algorithm to extract microsaccade-related neural activity from single-trial local field potential (LFP). We showed via extensive computer simulations that our approach can be used to reliably extract microsaccade-related activity. We then applied our method to real cortical LFP data collected from multiple visual areas of monkeys performing a generalized flash suppression task and demonstrated that our approach has excellent performance in extracting microsaccade-related signal from single-trial LFP data.     

视觉审美体验的类脑计算模型研究进展

目前神经科学与信息科学交叉发展的一个重要方向是面向高级认知功能的神经系统模型的构建,其标志着人类对自身特有的心智活动的研究进入到一个新阶段。简要综述了关于视觉审美体验的生理基础研究的进展,指出大脑神经回路是视觉审美体验的生理基础,其所涉及的大脑皮层和脑组织在视觉情感感知中功能的科学解析是构建类脑计算模型的基础。分别介绍了神经信息学、神经认知科学和神经美学框架下的视觉审美体验的类脑计算模型,并对模型进行了比较分析。对视觉审美体验的类脑计算模型研究的发展趋势进行了展望,阐述了该领域研究的意义。视觉审美体验类脑模型的构建有助于揭示大脑皮层和脑组织在视觉情感感知中的功能,有助于实现视觉情感体验的认知模拟。     

Eye movement as a mediator of the relationships among time pressure,feedback, and learning performance

The goal of this study is to examine the effects of time pressure and feedback on learning performance, as mediated by eye movement. Time pressure is one of main causes of human error in the workplace. Providing participants with feedback about their performance before task completion has been shown to reduce human error in diverse domains. Since both time pressure and feedback induce motivation, which is closely related to attention, we measured participants' eye movements to trace their attention and information acquisition coupled with a visual display. Time-to-deadline (long and short) and the presence of feedback were the independent factors used while measuring participants’ performance and eye movements as they learned new information about the subject of project management and answered multiple-choice questions via self-paced online learning systems. Using structural equation modeling, we found a mediating effect of eye movement on the relationships among time-to-deadline, feedback, and learning performance. Insufficient time-to-deadline accelerated the number of fixations on the screen, which resulted in longer task completion times and increased correct rates for participants learning about project management. The models in this study suggest the possibility of predicting performance from eye movement under time-to-deadline and feedback conditions. The structural equation model in the study can be applied to online and remote learning systems, in which time management is one of the main challenges for individual learners.     

视觉感知正反馈的显著性检测

目的 人类视觉系统性能远超当前机器视觉,模拟人类视觉机制改进当前算法是有效研究途径,为此提出一种视觉感知正反馈模型,通过循环迭代、重复叠加视觉刺激生成更符合人类感知的视觉显著性图。方法 首先用多种常规方法检测图像显著度,模拟人类视觉多通道特性,再组合这些显著图为综合显著图;利用显著度大的像素构建初始注视区。其次借助集成RVFL（随机向量功能网络）模拟人脑神经网络产生视觉刺激,对注视与非注视区内像素在线“随机采样—学习建模”,图像像素经模型分类获得新注视区。对新注视区与非注视区,可重复迭代进行“随机采样—学习建模—像素分类”;迭代中若注视区连续相同,则表明感知饱和,迭代终止。若将每次像素分类结果看做是一种视觉刺激,则多次视觉刺激输出叠加,可生成新的图像显著性图。最终的像素分类结果就是图像分割目标。结果 将本文算法与现有方法在标准图像数据库上进行对比评测,包括通过对6种算法在ECSSD、SED2和MSRA10K 3个图像数据库上的P-R曲线,F-measure值和平均绝对误差（MAE）值上进行定量分析,对6种模型生成的显著性图作定性比较。数据表明,本文算法在SED2和MSRA10K图象数据库中性能最好,在ECSSD图象数据库中稍低于BL（bootstrap learning）和RBD（robust background detection）算法。本文算法的显著图与人类视觉感知更接近。且算法的正反馈迭代过程一般可迅速饱和,并未显著增加算法负担。实验结果表明,本文方法可作为一种有效的后处理手段,显著提升常规显著性检测算法的性能。结论 提出了一种模拟人类视觉机制的数据驱动显著性检测算法,无需图像先验知识和事先的标记样本。面对多目标,背景复杂等情况,本文方法具有相对好的鲁棒性和适用性,并且能够较好解决现实环境中图像处理算法的通用性、可靠性和准确性问题。     

Eye movements predict students' computer-based assessment performance of physics concepts in different presentation modalities

Despite decades of studies on the link between eye movements and human cognitive processes, the exact nature of the link between eye movements and computer-based assessment performance still remains unknown. To bridge this gap, the present study investigates whether human eye movement dynamics can predict computer-based assessment performance (accuracy of response) in different presentation modalities (picture vs. text). Eye-tracking system was employed to collect 63 college students' eye movement behaviors while they are engaging in the computer-based physics concept questions presented as either pictures or text. Students' responses were collected immediately after the picture or text presentations in order to determine the accuracy of responses. The results demonstrated that students' eye movement behavior can successfully predict their computer-based assessment performance. Remarkably, the mean fixation duration has the greatest power to predict the likelihood of responding the correct physics concepts successfully, followed by re-reading time in proportion. Additionally, the mean saccade distance has the least and negative power to predict the likelihood of responding the physics concepts correctly in the picture presentation. Interestingly, pictorial presentations appear to convey physics concepts more quickly and efficiently than do textual presentations. This study adds empirical evidence of a prediction model between eye movement behaviors and successful cognitive performance. Moreover, it provides insight into the modality effects on students' computer-based assessment performance through the use of eye movement behavior evidence.     

Adaptation and learning in control of voluntary movement by the central nervous system

In order to control voluntary movements, the central nervous system must solve the following three computational problems at different levels: (1) the determination of a desired trajectory in visual coordinates; (2) the transformation of its coordinates into body coordinates; and (3) the generation of motor command. Concerning these problems, relevant experimental observations obtained in the field of neuroscience are briefly reviewed. On the basis of physiological information and previous models, we propose computational theories and a neural network model which account for these three problems. (1) A minimum torque-change model which predicts a wide range of trajectories in human multi-joint arm movements is proposed as a computational model for trajectory formation. (2) An iterative learning scheme is presented as an algorithm which solves the coordinate transformation and the control problem simultaneously. This algorithm can be regarded as a Newton-like method in function spaces. (3) A neural network model for generation of motor command is proposed. This model contains internal neural models of the motor system and its inverse system. The inverse-dynamics model is acquired by heterosynaptic plasticity using a feedback motor command (torque) as an error signal. The hierarchical arrangement of these neural networks and their global control are discussed. Their applications to robotics are also discussed.