基于动态贝叶斯网络的音视频双模态说话人识别

动态贝叶斯网络在描述具有多个通道的复杂随机过程方面具有优异的性能．基于动态贝叶斯网络进行音视频双模态说话人识别的工作．分析了音视频联合建模的层级结构，利用动态贝叶斯网络对不同层级的音视频关联关系建立模型，并基于该模型进行音视频说话人识别的实验．通过对不同层级的建模过程及说话人识别实验的结果进行分析，结果表明，动态贝叶斯网络为描述音视频间的时序相关性和特征相关性提供了有效的建模方法，在不同语音信噪比的情况下均能提高说话人识别的性能．

Audio-Visual Bimodal Speaker Identification Using Dynamic Bayesian Networks

Studied in this paper is the use of dynamic Bayesian networks (DBNs) for the task of text prompt audio-visual bimodal speaker identification. The task is to determine the identity of a speaker from a temporal sequence of audio and visual observations obtained from the acoustic speech and the shape of the mouth respectively. According to the hierarchical structure of audio-visual bimodal modeling, a new DBN is constructed to describe the natural audio and visual state asynchrony as well as their conditional dependency over time. The experimental results show that the dynamic Bayesian network is a powerful and flexible methodology for representing and modeling the audio-visual correlations and the proposed DBN can improve the accuracy of audio-only speaker identification at all levels of acoustic signal-to-noise ratio (SNR) from 0 to 30dB.