混合CTC/Attention模型在普通话识别中的应用

基于链接时序分类（Connectionist Temporal Classification, CTC）的端到端语音识别模型具有结构简单且能自动对齐的优点，但识别准确率有待进一步提高。本文引入注意力机制（Attention）构成混合CTC/Attention端到端模型，采用多任务学习方式，充分发挥CTC的对齐优势和Attention机制的上下文建模优势。实验结果表明，当选取80维FBank特征和3维pitch特征作为声学特征，选择VGG-双向长短时记忆网络（VGG-Bidirectional long short-time memory, VGG-BiLSTM）作为编码器应用于中文普通话识别时，该模型与基于CTC的端到端模型相比，字错误率下降约6.1%，外接语言模型后，字错误率进一步下降0.3%；与传统基线模型相比，字错误率也有大幅度下降。

Application of Hybrid CTC/Attention Model in Mandarin Recognition

The end-to-end speech recognition model based on Connectionist Temporal Classification （CTC） has the advantages of simple structure and automatic alignment, but the recognition accuracy needs to be further improved. This paper introduces the attention mechanism to form a hybrid CTC/Attention end-to-end model. This method adopts the multi-task learning approach, combining the alignment advantage of CTC with the context modeling advantage of attention mechanism. The experimental results show that when the 80-dimensional FBank feature and the 3-dimensional pitch feature are selected as the acoustic features, and the VGG-Bidirectional long short-time memory network is selected as the encoder for Chinese Mandarin recognition, the character error rate of this hybrid model is reduced by about 6.1% compared with the end-to-end model based on CTC, after the external language model is connected, the character error rate is further reduced by 0.3%. Compared with the traditional baseline model, the character error rate also decreased significantly.