Dynamic integration of height maps into a 3D world representation from range image sequences

Integration of 21/2D sketches obtained at different observation stations into a consistent world (or object) representation is one of the central issues in computer vision and robotics. The resolution and accuracy of 21/2D sketches may be different from one view point to another, and inconsistent data between different observations may occur. This article presents an approach to building a spatiotemporal representation of dynamic scenes including moving objects from a sequence of range images taken by a moving observer. A range image is transformed into a height-map representation, which is segmented into the ground plane and objects on it. In order to capture the resolution and accuracy of the range information and the consistency of the height information between different height maps, we define a reliability measure of the height information for each bucket on the height map. Using this reliability, the system finds the correspondences of both static and moving objects between different observations, and successively refines the height information and its reliability with newly acquired data, dealing with inconsistent data. Final representation of the integrated height map consists of the time stamp of the last observation, region labels of static and moving objects and their spatiotemporal properties such as height information, its reliability, and the velocities of both the observer and independently moving objects. We applied the method to road scenes physically simulated by landscape toy models and show the experimental results.