Multi-modal approach to evaluate adaptive visual stimuli of remote operation system using gaze behavior

A multi-modal approach is proposed to evaluate the usability of Adaptive Visual Stimuli for User Interface (AVS4UI) of remote operation systems. This study focuses on the evaluation of AVS4UI for forklift work because the operation complexity includes driving and cargo handling, which typically requires multiple salient attention. Presenting this amount of information simultaneously on a User Interface (UI) tends to cause confusions to operators and reduces operation efficiency. AVS4UI can therefore be one of the promising solutions where the optimal visual stimuli are autonomously presented for different work conditions. However, evaluation of AVS4UI is challenging because operators may be disoriented by adaptive information and worked without safety considerations. Therefore, novel gaze metrics are proposed to evaluate responses of forklift operators to AVS4UI so that undesired behavior can be evaluated. The proposed metrics implicitly represent gaze pattern in terms of transition and distribution between UI elements, operation safety, and familiarity with the adaptive system. The ideal AVS4UI is expected to minimize the proposed gaze metrics and outperform the non-adaptive UI. More importantly, the results of these metrics are consistent with those of perceived workload defined by NASA-Task Load Index. We also propose a correlation model using stepwise linear regression that provides reasonable estimation of perceived workload. Such novel metrics and correlation model enable objective and online evaluation to minimize biases of subjective response. Therefore, online work support system can be developed to support workers.