Shifting role for human factors in an ‘unmanned’ era

Human factors practitioners (HFPs) play many different roles in the design, creation, operation and maintenance of engineered systems. Less well known are the methods which are aimed at helping with the early stages of design, which are more systems-oriented and often involve questions of the concept of operation in which the engineered system will be fielded. Emerging from the field of cognitive engineering, these methods, including simulation, cognitive work analysis, cognitive task analyses and hierarchical task analysis, will be important as autonomous systems become increasingly capable. Even the most capable systems will continue to interact with humans, and it is at these interfaces between humans and engineered systems that HFP will continue to be needed. This paper describes recent work to leverage these methods to inform concepts of operation in aviation and space, machine learning algorithms and goal-oriented human–machine collaboration.     

Human-autonomy teaming in military settings

ABSTRACT

In this Thematic Issue ‘Human-Autonomy Teaming’ of Theoretical Issues in Ergonomics Science, five U.S. military-funded research efforts are presented to discuss human factors issues in a variety of military human-autonomy teaming mission environments: dismounted infantry working with a small ground robot; intelligence analysis; human working with an intelligent agent to manage a team of heterogeneous unmanned vehicles; vehicle-mounted ground penetrating radar. The research issues addressed in this Issue are diverse – from display designs to operator performance and trust in the systems. The results and insights documented in these five articles should provide useful resources to researchers and practitioners working on intelligent and autonomous systems.     

一种基于分层的人类动作识别和定位算法研究

人类动作识别在视频自动分析、视频检索等领域获得广泛应用,是目前的研究热点。然而现有的动作识别方法重点关注视频的非静态部分而忽略大部分静态部分,从而影响了动作识别和定位的效果。本文提出一种新的分层空间-时间分段表示法,以分层方式实现部位和整个身体的多分辨率表示,可用于运动识别和定位。该算法分为3个步骤。第一步,首先对每个视频帧进行分层分段,以得到一组分段树,每颗树是身体分段树的候选。第二步,利用视频的轮廓、接合对象结构、全局前景色等信息对候选分段树进行修剪。第三步,在时域上对剩余分段层的每个分段进行前向和后向跟踪。我们以难度较大的UCF-Sports和HighFive数据集为实验对象,对本文方法进行性能评估,实验结果表明,本文方法的性能要优于当前最新运动检测算法性能,运动定位性能与当前最新算法相当。     

Leveraging Transfer Learning for Spatio-Temporal Human Activity Recognition from Video Sequences

Human Activity Recognition (HAR) is an active research area due to its applications in pervasive computing, human-computer interaction, artificial intelligence, health care, and social sciences. Moreover, dynamic environments and anthropometric differences between individuals make it harder to recognize actions. This study focused on human activity in video sequences acquired with an RGB camera because of its vast range of real-world applications. It uses two-stream ConvNet to extract spatial and temporal information and proposes a fine-tuned deep neural network. Moreover, the transfer learning paradigm is adopted to extract varied and fixed frames while reusing object identification information. Six state-of-the-art pre-trained models are exploited to find the best model for spatial feature extraction. For temporal sequence, this study uses dense optical flow following the two-stream ConvNet and Bidirectional Long Short Term Memory (BiLSTM) to capture long-term dependencies. Two state-of-the-art datasets, UCF101 and HMDB51, are used for evaluation purposes. In addition, seven state-of-the-art optimizers are used to fine-tune the proposed network parameters. Furthermore, this study utilizes an ensemble mechanism to aggregate spatial-temporal features using a four-stream Convolutional Neural Network (CNN), where two streams use RGB data. In contrast, the other uses optical flow images. Finally, the proposed ensemble approach using max hard voting outperforms state-of-the-art methods with 96.30% and 90.07% accuracies on the UCF101 and HMDB51 datasets.     

Augmented body surveillance: Human microchip implantations and the omnipresent threat of function creep

Implanted microchips can store users' medical, financial, and other personal information, and provide users with easy and quick access to various locations and items. While adopted for their convenience outside of the healthcare sector, these invasive, semi-permanent implantable devices create augmented bodies that can be subject to ubiquitous surveillance. Situating human microchip implantations within surveillance literature, we draw from neoliberal perspectives of surveillance to examine augmented bodies, particularly as sources for market activity and as subjects of social control and sorting when these bodies are used as access control mechanisms, payment methods, and tracking means in employment, residential, commercial, and transportation sectors. History has demonstrated time and time again how unfettered technology applications and uses have led to real and/or perceived misuse by private and public sectors. Through the lens of function creep, we identify a pattern of expansion of applications and uses of technology beyond those originally intended across new technologies, such as DNA genetic genealogy databases, IoT wearables, and COVID-19 contact tracing apps, and provide illustrative examples of function creep, particularly the use of these technologies in criminal investigations and prosecutions despite not being intended or marketed for such use. By demonstrating the lack of clearly defined boundaries in the applications and uses of various new technologies and their associated data, and the ways they were misused, we demonstrate how human microchip implantations are headed on a similar path. The current and potential future uses of this technology raise concerns about the absence of regulation, law, and policy barring or limiting its application and use in specific sectors, and the impact of this technology on users’ security, data protection, and privacy. Undeniably, the present and potential future functions, applications, uses, and extensions of human microchip implantations in various sectors warrant a proactive examination of their security, privacy, and data protection consequences and the implementation of proactive policies to regulate new and currently unregulated uses of this technology and its associated data within these sectors.     

Towards the appropriate human-centred information systems: A case study of the Japanese retail industry

The industries of Japan have developed by learning from Western industries, especially the USA, and by implementing many of their concepts and technologies. However, Japanese industries have often implemented these concepts and technologies in a very different way from the USA. For example, while the USA uses information systems in retail industries as a tool by which data are collected and analysed to control the market, in Japan this same technology is considered rather as a learning device to interpret the market. While in the USA the market is seen as a natural phenomenon capable of being controlled, the Japanese see it as an ambiguous phenomenon that is ever changing and is not capable of being controlled. Rather it is important to feel the change in the market itself.This paper introduces human centredness to the information system, and argues against modern rationalism, i.e. human versus technology, taking the case of use of POS data from the POS system (point of sale: a system that collects data on both the customer and goods sold by scanning bar codes that are attached to the surface of the goods) by the eminent Japanese retailer, Ito-Yokado. It emphasises an interactive concept of interaction between human and technology of the postmodern paradigm.     

基于云计算的远距护理服务研究

近年来网络技术发展迅速,Web2.0概念已经普遍引入在各大网站,平台应用也从个人计算机转移到行动装置上,重视以人为主的Human 2.0风潮正逐渐兴起,包括触控、时尚化设计、行动运用和人类主导的科技趋势,已取代过去一味追求科技进步,而回到解决人类问题、面对人类原始能力为出发点;而要能储存大量的健康指针与生理讯号并提供实时的服务,云运算(Cloud Compu-ting)环境已获得证实较为有效且扩充弹性大;文章提出一个基于云运算环境运用于健康护理整合服务平台架构,底层结合WSN感测网络撷取生理参数,经由Internet上传到云系统平台,让使用者以透过行动装置监控个人健康信息,文章提出的技术平台可减少后端环境布署的建置成本与维护费用约1/3,系统能够方便动态地拓展与布署,使其资源运用最佳化,另外利用云强大的运算能力可减少Client端硬件资源损耗可以不消耗大80%以上的系统资源,系统还能承载多个用户存取,并维持服务稳定运作。     

Hands-free administration of subjective workload scales: Acceptability in a surgical training environment

Introduction

Subjective workload measures are usually administered in a visual-manual format, either electronically or by paper and pencil. However, vocal responses to spoken queries may sometimes be preferable, for example when experimental manipulations require continuous manual responding or when participants have certain sensory/motor impairments. In the present study, we evaluated the acceptability of the hands-free administration of two subjective workload questionnaires - the NASA Task Load Index (NASA-TLX) and the Multiple Resources Questionnaire (MRQ) - in a surgical training environment where manual responding is often constrained.

Method

Sixty-four undergraduates performed fifteen 90-s trials of laparoscopic training tasks (five replications of 3 tasks - cannulation, ring transfer, and rope manipulation). Half of the participants provided workload ratings using a traditional paper-and-pencil version of the NASA-TLX and MRQ; the remainder used a vocal (hands-free) version of the questionnaires. A follow-up experiment extended the evaluation of the hands-free version to actual medical students in a Minimally Invasive Surgery (MIS) training facility.

Results

The NASA-TLX was scored in 2 ways - (1) the traditional procedure using participant-specific weights to combine its 6 subscales, and (2) a simplified procedure - the NASA Raw Task Load Index (NASA-RTLX) - using the unweighted mean of the subscale scores. Comparison of the scores obtained from the hands-free and written administration conditions yielded coefficients of equivalence of r = 0.85 (NASA-TLX) and r = 0.81 (NASA-RTLX). Equivalence estimates for the individual subscales ranged from r = 0.78 (“mental demand”) to r = 0.31 (“effort”). Both administration formats and scoring methods were equally sensitive to task and repetition effects. For the MRQ, the coefficient of equivalence for the hands-free and written versions was r = 0.96 when tested on undergraduates. However, the sensitivity of the hands-free MRQ to task demands (η_partial² = 0.138) was substantially less than that for the written version (η_partial² = 0.252). This potential shortcoming of the hands-free MRQ did not seem to generalize to medical students who showed robust task effects when using the hands-free MRQ (η_partial² = 0.396). A detailed analysis of the MRQ subscales also revealed differences that may be attributable to a “spillover” effect in which participants’ judgments about the demands of completing the questionnaires contaminated their judgments about the primary surgical training tasks.

Conclusion

Vocal versions of the NASA-TLX are acceptable alternatives to standard written formats when researchers wish to obtain global workload estimates. However, care should be used when interpreting the individual subscales if the object is to make comparisons between studies or conditions that use different administration modalities. For the MRQ, the vocal version was less sensitive to experimental manipulations than its written counterpart; however, when medical students rather than undergraduates used the vocal version, the instrument’s sensitivity increased well beyond that obtained with any other combination of administration modality and instrument in this study. Thus, the vocal version of the MRQ may be an acceptable workload assessment technique for selected populations, and it may even be a suitable substitute for the NASA-TLX.     

The effect of sub-threshold forces on human performance in multi-modal computer-aided design

In traditional CAD (computer-aided design) systems, the manipulation of points and lines is often difficult because designers manipulate virtual objects through their vision system. Nowadays, designers can explore and manipulate virtual objects in haptic-enabled CAD systems using haptic devices. Haptic devices can present force feedbacks to pull or push the users’ hands into desirable targets. Of course the intent is for the user to experience the same sensations in the virtual realm as they would in the real world. Thus, sub-threshold forces, which cannot be perceived by users, should be incorporated in the control of users’ movements. As a result, our attention is directed to study the effect of sub-threshold forces on the accuracy of movement in a haptic-enabled virtual reality (VR) system. In this study, our goal is to manipulate users’ hands using controlled forces such that users cannot notice the forces. With this in mind, we have constructed a haptic-enabled virtual environment (VE) to carry out a multi-modal Fitts’ type task. In the task, subjects could see the position of the haptic probe in the VE where forces were applied on their hands. Basically, the accuracy of subjects was measured using a performance index when the intensity and direction of forces changed. A psychophysical method was utilized to ensure that the forces were below the force threshold of the human force perception. Results indicate that the accuracy is affected by the intensity and direction of sub-threshold forces even when users are allowed to control their actions through visual feedbacks.     

Associating visual textures with human perceptions using genetic algorithms

This paper deals with an approach allowing to associate visual textures with given human perceptions. Hereby, based on a forward model associating human perceptions for given visual textures, the deduction of an reverse process is presented which is able to associate and characterize visual textures for given human perceptions. For doing so, we propose a constraint-based genetic algorithm approach, which is able to minimize a specific optimization problem containing constraints in form of band-widths for valid individuals (low level features extracted from textures) in a population. The constraints are determined by relationships between (low level) features characterizing textures in form of high-dimensional approximation models. Additionally, in each iteration step checking for valid individuals is carried out with a texture/non-texture classifier or by using a convex hull over a set of valid textures. The whole approach is evaluated based on a real-world texture set used as a start population in the genetic algorithm and by defining various kinds of human perceptions (for which textures are sought) represented by adjective vectors in the aesthetic space. The generated individuals (low level feature vectors) have a high level of fitness (they are quite close to the pre-defined adjective vectors) and a small distance to the initial population. The textures synthesized based on the generated individuals are visualized and compared with textures synthesized by a time-intensive direct texture mixing and re-combination method based on a real-world texture data base.