Human visual sensitivity based optimal local backlight dimming methodologies under different viewing conditions

An adaptive local backlight dimming algorithm is proposed considering human visual sensitivities under various viewing conditions. Local dimming amount is maximized based on contrast sensitivity function and local image characteristics to minimize perceived luminance and contrast losses maintaining image qualities. Average luminance, contrast and spatial frequency are evaluated as local image characteristics. As dimming based on local characteristics has blocking artifacts, a compensation method using edge luminance difference is proposed considering human responses. The proposed algorithms reduce backlight power consumption by 39.69% on average, with an average SSIM of 0.995. It can be increased by 5.66% with varying viewing conditions, resulting in a 45.35% power consumption reduction on average. The outperformances of the proposed algorithm in power consumption reduction and maintaining image quality are verified with various image quality metrics compared to recent dimming algorithms.     

Difficulty and complexity definitions for assembly task allocation and assignment in human–robot collaborations: A review

This paper presents a literature review on the different aspects of task allocation and assignment problems in human–robot collaboration (HRC) tasks in industrial assembly environments. In future advanced industrial environments, robots and humans are expected to share the same workspace and collaborate to efficiently achieve shared goals. Difficulty- and complexity-aware HRC assembly is necessary for human-centric manufacturing, which is a goal of Industry 5.0. Therefore, the objective of this study is to clarify the definitions of difficulty and complexity used to encourage effective collaboration between humans and robots to leverage the adaptability of humans and the autonomy of robots. To achieve this goal, a systematic review of the following relevant databases for computer science was performed: IEEE Xplore, ScienceDirect, SpringerLink, ACM Digital Library, and ASME Digital Collection. The results extracted from 74 peer-reviewed research articles published until July 2022 were summarized and categorized into four taxonomies for 145 difficulty and complexity definitions from the perspectives of (1) definition-use objectives, (2) evaluation objectives, (3) evaluation factors, and (4) evaluation variables. Next, existing definitions were primarily classified according to the following two criteria to identify potential future studies on the formulation of new definitions for human-centric manufacturing: (1) agent specificity and (2) common aspects in manual and robotic assemblies.     

A state-of-the-art survey on Augmented Reality-assisted Digital Twin for futuristic human-centric industry transformation

The combination of Augmented Reality (AR) and Digital Twin (DT) has begun to show its potential nowadays, leading to a growing research interest in both academia and industry. Especially under the current human-centric trend, AR embraces the potential to integrate operators into the new generation of Human Cyber–Physical System (HCPS), in which DT is a pillar component. Some review articles have focused on this topic and discussed the benefits of combining AR and DT, but all of them are limited to a specific domain. To fill the gap, this research conducts a state-of-the-art survey (till 17-July-2022) from the AR-assisted DT perspective across different sectors of the industrial field, covering a total of 118 selected publications. Firstly, application scenarios and functions of AR-assisted DT are summarized by following the engineering lifecycle, among which production process, service design, and Human–Machine Interaction (HMI) are hot topics. Then, improvements specifically brought by AR are analyzed according to three dimensions, namely virtual twin, hybrid twin, and cognitive twin, respectively. Finally, challenges and future perspectives of AR-assisted DT for futuristic human-centric industry transformation are proposed, including promoting product design, robotic-related works, cyber–physical interaction, and human ergonomics.     

From tabular data to knowledge graphs: A survey of semantic table interpretation tasks and methods

Tabular data often refers to data that is organized in a table with rows and columns. We observe that this data format is widely used on the Web and within enterprise data repositories. Tables potentially contain rich semantic information that still needs to be interpreted. The process of extracting meaningful information out of tabular data with respect to a semantic artefact, such as an ontology or a knowledge graph, is often referred to as Semantic Table Interpretation (STI) or Semantic Table Annotation. In this survey paper, we aim to provide a comprehensive and up-to-date state-of-the-art review of the different tasks and methods that have been proposed so far to perform STI. First, we propose a new categorization that reflects the heterogeneity of table types that one can encounter, revealing different challenges that need to be addressed. Next, we define five major sub-tasks that STI deals with even if the literature has mostly focused on three sub-tasks so far. We review and group the many approaches that have been proposed into three macro families and we discuss their performance and limitations with respect to the various datasets and benchmarks proposed by the community. Finally, we detail what are the remaining scientific barriers to be able to truly automatically interpret any type of tables that can be found in the wild Web.     

Performance evaluation of a tactile and kinesthetic finger feedback system for teleoperation

Teleoperation during a catastrophic event requires an interface that can perform under frequently changing circumstances caused by unpredictable and dangerous conditions. Thus, teleoperation interfaces are under active development to provide both visual and haptic feedback to the fingers. However, studies of teleoperation systems with finger haptic feedback based on force profiles are difficult to conduct because of interface limitations. Therefore, in this paper, we introduce an intuitive teleoperation interface, an anthropomorphic teleoperated robot, and a hand-wearable force-feedback system that provides various feedbacks to the fingers. We combined these systems to compare and evaluated the performance of tactile and kinesthetic finger feedback using two experiments: maintaining appropriate grip force for variably fragile objects and following a force trajectory that changed in real time. Ten subjects participated in the experiments. The results were analyzed using repeated measures analysis of variance. Feedback factors differed significantly. Provision of force feedback to the user’s finger was most effective in both teleoperation experiments.     

Shifting role for human factors in an ‘unmanned’ era

Human factors practitioners (HFPs) play many different roles in the design, creation, operation and maintenance of engineered systems. Less well known are the methods which are aimed at helping with the early stages of design, which are more systems-oriented and often involve questions of the concept of operation in which the engineered system will be fielded. Emerging from the field of cognitive engineering, these methods, including simulation, cognitive work analysis, cognitive task analyses and hierarchical task analysis, will be important as autonomous systems become increasingly capable. Even the most capable systems will continue to interact with humans, and it is at these interfaces between humans and engineered systems that HFP will continue to be needed. This paper describes recent work to leverage these methods to inform concepts of operation in aviation and space, machine learning algorithms and goal-oriented human–machine collaboration.     

Human-autonomy teaming in military settings

ABSTRACT

In this Thematic Issue ‘Human-Autonomy Teaming’ of Theoretical Issues in Ergonomics Science, five U.S. military-funded research efforts are presented to discuss human factors issues in a variety of military human-autonomy teaming mission environments: dismounted infantry working with a small ground robot; intelligence analysis; human working with an intelligent agent to manage a team of heterogeneous unmanned vehicles; vehicle-mounted ground penetrating radar. The research issues addressed in this Issue are diverse – from display designs to operator performance and trust in the systems. The results and insights documented in these five articles should provide useful resources to researchers and practitioners working on intelligent and autonomous systems.     

一种基于分层的人类动作识别和定位算法研究

人类动作识别在视频自动分析、视频检索等领域获得广泛应用,是目前的研究热点。然而现有的动作识别方法重点关注视频的非静态部分而忽略大部分静态部分,从而影响了动作识别和定位的效果。本文提出一种新的分层空间-时间分段表示法,以分层方式实现部位和整个身体的多分辨率表示,可用于运动识别和定位。该算法分为3个步骤。第一步,首先对每个视频帧进行分层分段,以得到一组分段树,每颗树是身体分段树的候选。第二步,利用视频的轮廓、接合对象结构、全局前景色等信息对候选分段树进行修剪。第三步,在时域上对剩余分段层的每个分段进行前向和后向跟踪。我们以难度较大的UCF-Sports和HighFive数据集为实验对象,对本文方法进行性能评估,实验结果表明,本文方法的性能要优于当前最新运动检测算法性能,运动定位性能与当前最新算法相当。     

Leveraging Transfer Learning for Spatio-Temporal Human Activity Recognition from Video Sequences

Human Activity Recognition (HAR) is an active research area due to its applications in pervasive computing, human-computer interaction, artificial intelligence, health care, and social sciences. Moreover, dynamic environments and anthropometric differences between individuals make it harder to recognize actions. This study focused on human activity in video sequences acquired with an RGB camera because of its vast range of real-world applications. It uses two-stream ConvNet to extract spatial and temporal information and proposes a fine-tuned deep neural network. Moreover, the transfer learning paradigm is adopted to extract varied and fixed frames while reusing object identification information. Six state-of-the-art pre-trained models are exploited to find the best model for spatial feature extraction. For temporal sequence, this study uses dense optical flow following the two-stream ConvNet and Bidirectional Long Short Term Memory (BiLSTM) to capture long-term dependencies. Two state-of-the-art datasets, UCF101 and HMDB51, are used for evaluation purposes. In addition, seven state-of-the-art optimizers are used to fine-tune the proposed network parameters. Furthermore, this study utilizes an ensemble mechanism to aggregate spatial-temporal features using a four-stream Convolutional Neural Network (CNN), where two streams use RGB data. In contrast, the other uses optical flow images. Finally, the proposed ensemble approach using max hard voting outperforms state-of-the-art methods with 96.30% and 90.07% accuracies on the UCF101 and HMDB51 datasets.     

Augmented body surveillance: Human microchip implantations and the omnipresent threat of function creep

Implanted microchips can store users' medical, financial, and other personal information, and provide users with easy and quick access to various locations and items. While adopted for their convenience outside of the healthcare sector, these invasive, semi-permanent implantable devices create augmented bodies that can be subject to ubiquitous surveillance. Situating human microchip implantations within surveillance literature, we draw from neoliberal perspectives of surveillance to examine augmented bodies, particularly as sources for market activity and as subjects of social control and sorting when these bodies are used as access control mechanisms, payment methods, and tracking means in employment, residential, commercial, and transportation sectors. History has demonstrated time and time again how unfettered technology applications and uses have led to real and/or perceived misuse by private and public sectors. Through the lens of function creep, we identify a pattern of expansion of applications and uses of technology beyond those originally intended across new technologies, such as DNA genetic genealogy databases, IoT wearables, and COVID-19 contact tracing apps, and provide illustrative examples of function creep, particularly the use of these technologies in criminal investigations and prosecutions despite not being intended or marketed for such use. By demonstrating the lack of clearly defined boundaries in the applications and uses of various new technologies and their associated data, and the ways they were misused, we demonstrate how human microchip implantations are headed on a similar path. The current and potential future uses of this technology raise concerns about the absence of regulation, law, and policy barring or limiting its application and use in specific sectors, and the impact of this technology on users’ security, data protection, and privacy. Undeniably, the present and potential future functions, applications, uses, and extensions of human microchip implantations in various sectors warrant a proactive examination of their security, privacy, and data protection consequences and the implementation of proactive policies to regulate new and currently unregulated uses of this technology and its associated data within these sectors.