基于生理舌头模型的语音可视化系统

目的 目前针对舌头的语音同步动画技术还未得到广泛的研究。在此背景下,提出了一种基于生理模型的舌头动画合成方法。方法 首先构建了一个精细的、能够在肌肉激励下产生逼真舌头变形的舌头生理模型;其次利用该舌头模型合成了大量的舌头运动样本,并据此通过学习得到一个从肌肉激励到舌头轮廓的转换模型;然后对采集的动态2维舌头轮廓数据进行运动参数估计以得到与音素对应的体素(肌肉激励序列和刚体位移序列);最后将体素按一定的排列方式输入到舌头生理模型进行仿真以生成相应的舌头动画。结果 该系统可以合成听觉效果逼真的语音和视觉效果逼真且与合成语音同步的舌头动画。结论 本文方法可以根据汉语普通话或其他语言的2维舌头轮廓数据构建音素—体素数据库,并据此合成该语言对应的高真实感的3维舌头动画。

Speech visualization system based on physiological tongue model

Objective Speech synchronized tongue animation remains lacking in research. Under this background, this paper proposes a physiology-based tongue animation system. Method First, an accurate physiology-based tongue model is created, the deformation of which can be driven by muscle activations. Second, the model is utilized to produce numerous tongue deformation samples according to numerous designed muscle activations. With these samples, a neural network that can transform muscle activations to tongue deformation is trained. Then, from the 2D tongue deformation results on tongue X-ray data, the corresponding physemes (muscle activation and rigid movement sequences) are estimated with this neural network. Lastly, speech synchronized tongue animation is synthesized by inputting these physemes into the tongue model for simulation. Result Experiment results demonstrate that the proposed system can produce realistic-sounding voices and visually realistic speech synchronized tongue animation. Conclusion The system can be used to build a phonemes-physemes database from collected 2D tongue movement data on Mandarin Chinese or other languages and can synthesize highly realistic tongue animation corresponding to the language.