This paper designs a visual-inertial odometry based on a depth camera and an inertial sensor for localizing the camera. The odometry contains camera localization and camera relocalization. Camera localization uses the invariant extended Kalman filter (IEKF) to fuse multilevel iteration closest point (ICP) estimates with the measurements from the inertial sensor for obtaining an accurate camera pose, in which, the estimated error of the multilevel ICP is quantized by the fisher information matrix. Because massive points are set as the inputs, GPU parallel computing is used to fast implement multilevel ICP estimation and its error quantization. When the odometry tracks camera failed, a constant velocity model is constructed with the data from the inertial sensor, and the random fern method is improved based on the velocity model to relocate the odometry. The experiment results show that the designed odometry can track the camera accurately and relocalize the camera effectively.