基于混合式注意力机制的语音识别研究

为了解决语音识别中基于卷积位置信息的混合式注意力机制无法提取长期有效位置信息的问题，提出了一种捕捉长期有效位置信息的新型混合式注意力机制。首先，对当前时刻生成的注意力得分作卷积来提取多通道特征图，并通过全局平均池化来得到恒定维度的特征向量；接着，引入长短期记忆网络（long short-term memory，LSTM）单元作为外部记忆模块，并以生成的特征向量作为输入，生成下一时刻的位置信息向量；最后，结合经典的LAS（listen，attend and spell）模型来验证提出方案的有效性。实验结果表明，该方案能充分考虑过去多个时刻的注意力得分。相对于基于卷积位置信息的LAS模型，该方案在纯净和含噪语音数据集上取得的标签错误率分别减少了1.8%和2.21%。

Research on speech recognition based on hybrid attention mechanism

In speech recognition, the convolution location-based hybrid attention mechanism can not extract location information that can be valid over long term. This paper proposed a new hybrid attention mechanism to solve this problem. Firstly it convolved with the attention score generated for the current time to extract multi-channel features, followed by obtaining the feature vectors of constant dimensions via global average pooling. Then it introduced a LSTM unit as the external memory module and used the generated feature vectors as input to generate the location vectors for the next time point. Finally this paper used the classic LAS model to verify the effect of the new hybrid attention mechanism. Experiment results show that the new hybrid attention mechanism can take full consideration of the attention scores at many past time points. Compared to the convolution location-based LAS model, the label error rate of the proposed scheme on pure and noisy speech datasets is reduced by 1.8% and 2.21%, respectively.