Hybrid machine learning for human action recognition and prediction in assembly |
| |
| Affiliation: | 1. Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH 44106, USA;2. Department of Electrical and Computer Engineering, Department of Mechanical Engineering, University of Kentucky, Lexington, KY 40506, USA;1. School of Mechanical Engineering, Northwestern Polytechnical University, Xi''an 710072, China;2. Department of Mechanical Engineering, Faculty of Engineering, National University of Singapore, 117576, Singapore;1. Laboratory for Manufacturing Systems and Automation, Department of Mechanical Engineering and Aeronautics, University of Patras, Patras 26504, Greece;1. Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden;1. Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, United States;2. Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden;3. Department of Mechanical and Aerospace Engineering, University of Virginia, Charlottesville, VA, United States;1. College of Mechanical Engineering, Donghua University, Shanghai, 201620, China;2. Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, China;3. Department of Mechanical Engineering, The University of Auckland, 0632, New Zealand |
| |
| Abstract: | As one of the critical elements for smart manufacturing, human-robot collaboration (HRC), which refers to goal-oriented joint activities of humans and collaborative robots in a shared workspace, has gained increasing attention in recent years. HRC is envisioned to break the traditional barrier that separates human workers from robots and greatly improve operational flexibility and productivity. To realize HRC, a robot needs to recognize and predict human actions in order to provide assistance in a safe and collaborative manner. This paper presents a hybrid approach to context-aware human action recognition and prediction, based on the integration of a convolutional neural network (CNN) and variable-length Markov modeling (VMM). Specifically, a bi-stream CNN structure parses human and object information embedded in video images as the spatial context for action recognition and collaboration context identification. The dependencies embedded in the action sequences are subsequently analyzed by a VMM, which adaptively determines the optimal number of current and past actions that need to be considered in order to maximize the probability of accurate future action prediction. The effectiveness of the developed method is evaluated experimentally on a testbed which simulates an assembly environment. High accuracy in both action recognition and prediction is demonstrated. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
