Inserting virtual pedestrians into pedestrian groups video with behavior consistency

In this paper, we propose a novel approach to integrate virtual pedestrians into a scene of real pedestrian groups with behavior consistency, and this is achieved by dynamic path planning of virtual pedestrians. Rather than accounting for the local collision avoidance only, our approach is capable of finding an optimized path for each virtual pedestrian on his way based on the current global distribution of the real groups in the scene. The big challenge is due to the information of both position and velocity of real pedestrians in the video being unavailable; also the distribution of the groups in the scene may vary dynamically. We therefore need to detect and track real pedestrians on each frame of the video to acquire their distribution and motion information. We save this information by an efficient data structure, called environment grid. During the way of a virtual pedestrian, the respective agent frequently emits the detection rays through the environment cells to find the situation of the real pedestrians ahead of him and adjust the original path if necessary. Virtual pedestrians are merged into the video finally with the occlusion between virtual characters and the real pedestrians correctly presented. Experiment results on several scenarios demonstrate the effectiveness of the proposed approach.