航天员虚拟交互操作训练多体感融合驱动方法研究

针对航天员虚拟训练中的人机自然交互问题，基于体态/手势识别和人体运动特性，提出一种多通道数据融合的虚拟驱动与交互方法。结合Kinect 设备能够完整识别人体姿态特点及LeapMotion 设备能精确识别手势姿态的优势，提出了基于判断的数据传递方法，在人体关节识别的基础上对手部关节进行识别与数据处理计算，采用多通道体感识别融合方法将二者结合，并进行了实验。结果表明，通过采用LeapMotion 和Kinect 对手部识别的判别，当手势在LeapMotion 识别范围内，能够在实现人体体感识别的基础上增加较为精确的手势识别。此方法成功实现了人体姿态识别和手势精确识别的结合，可应用于航天员虚拟训练中的人机自然交互中去。

On Multi-Somatosensory Driven Method for Virtual Interactive Operation Training of Astronaut

To solve the problem of human-computer natural interaction in the virtual training ofastronauts, a multi-somatosensory driven method is proposed based on posture / gesture recognitionand human motion characteristics. With the the advantages of Kinect device which can completelyrecognize human posture characteristics and LeapMotion device which can accurately identifygestures, the method of data transfer based on judgement is put forward. Hand joints are recognizedand the related data are processed and calculated on the basis of the recognition of joints of the wholebody. These two are combined by using the multi-somatosensory driven method, and the experimentis carried out. The results show that by using LeapMotion and Kinect to recognize hand joints, whenthe gesture is within the range of LeapMotion recognition, we can add more precise gesturerecognition to the realization of human somatosensory recognition. This method has successfullyrealized the combination of human posture recognition and precise gesture recognition, and can beapplied to the human-computer natural interaction in the virtual training of astronauts.