Detecting spatiotemporal irregularities in videos via a 3D convolutional autoencoder |
| |
| Affiliation: | 1. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Ave., Singapore 639798, Singapore;2. Computer Science and Engineering Department, University at Buffalo, the State University of New York, United States;3. State Key Lab of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Ruoyu Road 129, Wuhan, China;1. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China;2. School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China;3. Institute of Information and Control, Hangzhou Dianzi University, Hangzhou 310018, China;1. Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an 710062, China;2. Engineering Laboratory of Teaching Information Technology of Shaanxi Province, Xi’an 710119, China;3. School of Computer Science, Shaanxi Normal University, Xi’an 710119, China;4. School of Computer Science and Technology, Nanjing Normal University, 210023, China;1. School of Computer Science, McGill University, Montreal, Canada;2. Department of Computer Science, Rutgers University, NJ, USA;3. Indraprastha Institute of Information Technology, New Delhi, India;4. Indian Institute of Technology, Kerala, India;1. Department of Electronics and Communication Engineering, VV College of Engineering, Tirunelveli, India;2. Department of Electronics and Communication Engineering, Thiagarajar College of Engineering, Madurai, India;1. Department of Information and Communication Engineering, Xiamen University, Xiamen, Fujian 361005, PR China;2. Shenzhen Research Institute of Xiamen University, Shenzhen, Guangdong 518057, PR China |
| |
| Abstract: | Spatiotemporal irregularities (i.e., the uncommon appearance and motion patterns) in videos are difficult to detect, as they are usually not well defined and appear rarely in videos. We tackle this problem by learning normal patterns from regular videos, while treating irregularities as deviations from normal patterns. To this end, we introduce a 3D fully convolutional autoencoder (3D-FCAE) that is trainable in an end-to-end manner to detect both temporal and spatiotemporal irregularities in videos using limited training data. Subsequently, temporal irregularities can be detected as frames with high reconstruction errors, and irregular spatiotemporal patterns can be detected as blurry regions that are not well reconstructed. Our approach can accurately locate temporal and spatiotemporal irregularities thanks to the 3D fully convolutional autoencoder and the explored effective architecture. We evaluate the proposed autoencoder for detecting irregular patterns on benchmark video datasets with weak supervision. Comparisons with state-of-the-art approaches demonstrate the effectiveness of our approach. Moreover, the learned autoencoder shows good generalizability across multiple datasets. |
| |
| Keywords: | Spatiotemporal irregularity detection Autoencoder 3D convolution Anomaly detection Unsupervised learning Real-time |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
