基于多头注意力机制的端到端语音情感识别

针对语音情感数据集规模小且数据维度高的特点，为解决传统循环神经网络（RNN）长程依赖消失和卷积神经网络（CNN）关注局部信息导致输入序列内部各帧之间潜在关系没有被充分挖掘的问题，提出一个基于多头注意力（MHA）和支持向量机（SVM）的神经网络MHA-SVM用于语音情感识别（SER）。首先将原始音频数据输入MHA网络来训练MHA的参数并得到MHA的分类结果；然后将原始音频数据再次输入到预训练好的MHA中用于提取特征；最后通过全连接层后使用SVM对得到的特征进行分类获得MHA-SVM的分类结果。充分评估MHA模块中头数和层数对实验结果的影响后，发现MHA-SVM在IEMOCAP数据集上的识别准确率最高达到69.6%。实验结果表明同基于RNN和CNN的模型相比，基于MHA机制的端到端模型更适合处理SER任务。

End-to-end speech emotion recognition based on multi-head attention

Aiming at the characteristics of small size and high data dimensionality of speech emotion datasets， to solve the problem of long-range dependence disappearance in traditional Recurrent Neural Network （RNN） and insufficient excavation of potential relationship between frames within the input sequence because of focus on local information of Convolutional Neural Network （CNN）， a new neural network MAH-SVM based on Multi-Head Attention （MHA） and Support Vector Machine （SVM） was proposed for Speech Emotion Recognition （SER）. First， the original audio data were input into the MHA network to train the parameters of MHA and obtain the classification results of MHA. Then， the same original audio data were input into the pre-trained MHA again for feature extraction. Finally， these obtained features were fed into SVM after the fully connected layer to obtain classification results of MHA-SVM. After fully evaluating the effect of the heads and layers in the MHA module on the experimental results， it was found that MHA-SVM achieved the highest recognition accuracy of 69.6% on IEMOCAP dataset. Experimental results indicate that the end-to-end model based on MHA mechanism is more suitable for SER tasks compared with models based on RNN and CNN.