基于双LSTM融合的类人机器人实时表情再现方法

为提高机器人表情再现的时空相似度和运动平滑度,结合序列到序列的深度学习模型,提出一种基于双LSTM(长短期记忆)融合的类人机器人实时表情再现方法.在离线机械建模阶段,首先构建逆向机械模型以实现面部特征序列到电机控制序列的逆向映射,并进一步提出运动趋势模型以规整电机连续运动的平滑度;然后,引入加权目标函数以实现两模型的融合和参数优化.在在线表情迁移阶段,以表演者面部特征序列作为融合模型的输入,并在最优参数下完成表演者面部特征序列到机器人控制序列的端-端翻译,从而达到机器人表情的帧-帧再现.实验结果表明:融合模型的电机控制偏差不超过8%,且表情再现的时空相似度和运动平滑度大于85%.与相关方法相比,提出的方法在控制偏差、时空相似度和运动平滑度方面均有较大提升.

A Real-time Expression Mimicking Method for Humanoid Robot Based on Dual LSTM Fusion

To improve space-time similarity and motion smoothness in robot expression imitation, a real-time expression mimicking method for humanoid robot based on dual LSTM(long short-term memory) fusion is proposed by combining with the sequence to sequence deep learning model. In offline mechanical modeling phase, an inverse mechanical model is constructed firstly to fulfill the inverse mapping from facial feature sequence to motor control sequence, and a motion tendency model is further presented to wrap the smoothness of continuous motor motion. Secondly, a weighted objective function is addressed to implement the fusion of the above two models as well as the parameter optimization. In online expression transfer phase, the facial feature sequence of performer is regarded as the input of the fusion model, and the frame-to-frame expression mimicking of robot is achieved by means of the end-to-end translation from the performer facial feature sequence to robot control sequence under the optimal parameters. The experimental results indicate that the control deviations of the fusion model is lower than 8%, meanwhile, the space-time similarity and motion smoothness in expression mimicking is greater than 85%. Compared with related methods, the proposed method has a significant improvement in the control deviation, space-time similarity and motion smoothness.