Multimodal and ontology-based fusion approaches of audio and visual processing for violence detection in movies |
| |
| Authors: | Thanassis Perperis Theodoros Giannakopoulos Alexandros Makris Dimitrios I Kosmopoulos Sofia Tsekeridou Stavros J Perantonis Sergios Theodoridis |
| |
| Affiliation: | 1. Dept. of Informatics and Telecommunications, University of Athens, GR 15784, Greece;2. NCSR Demokritos, Inst. of Informatics and Telecommunications, GR 15310, Greece;1. UMR CNRS 7253 Heudiasyc, Sorbonne Universités, Université de Technologie de Compiègne, CS 60319 – 60203 Compiègne cedex, France;2. Université de Picardie Jules Verne, France;1. Chief Resident, Submitted during Third Year Surgical Residency, Department of Podiatric Surgery, Oakwood Annapolis Hospital, Wayne, MI;2. Program Director, Podiatric Surgical Residency Program, Oakwood Annapolis Hospital, Wayne, MI;3. Research Director, Curriculum and Evaluation Director, Medical Education, Oakwood Hospital and Medical Center, Dearborn, MI;1. University of Rome “Sapienza”, Via Ariosto 25, Rome, 00185, Italy;2. University e-Campus, Via Isimbardi 10, Novedrate, 22060, Italy |
| |
| Abstract: | In this paper we present our research results towards the detection of violent scenes in movies, employing advanced fusion methodologies, based on learning, knowledge representation and reasoning. Towards this goal, a multi-step approach is followed: initially, automated audio and visual analysis is performed to extract audio and visual cues. Then, two different fusion approaches are deployed: (i) a multimodal one that provides binary decisions on the existence of violence or not, employing machine learning techniques, (ii) an ontological and reasoning one, that combines the audio-visual cues with violence and multimedia ontologies. The latter reasons out not only the existence of violence or not in a video scene, but also the type of violence (fight, screams, gunshots). Both approaches are experimentally tested, validated and compared for the binary decision problem of violence detection. Finally, results for the violence type identification are presented for the ontological fusion approach. For evaluation purposes, a large dataset of real movie data has been populated. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
