Sensor fusion based head pose tracking for lightweight flight cockpit systems

Flight cockpit head tracking systems (HTS) are one of the most important impetuses for head pose tracking in the field of augmented reality. For the purpose of implementing the natural interaction between pilot and the complete internal environment in our lightweight flight cockpit system, a head tracking system consisting of inside-out tracking (IOT) and outside-in tracking (OIT) is designed and a novel approach using sensor fusion is proposed to dynamically track pilot’s head pose. The proposed approach utilizes a sensor fusion framework, composed of extended Kalman filters and fusion filter, to fuse the poses from complementary IOT and OIT. An experimental setup is established to simulate the cockpit HTS and verify the proposed approach. Experimental results show that the proposed tracking scheme based on sensor fusion is capable of achieving more accurate and stable pose outputs, extending tracking range as well as better robustness compared with single IOT or OIT.