基于单向Transformer和孪生网络的多轮任务型对话技术

循环神经网络和Transformer在多轮对话系统的建模上依赖大量的样本数据且回复准确率过低.为此,提出一种针对任务型对话系统的建模方法.引入预训练模型对句子语意和对话过程进行深度编码,对Transformer模型进行精简,仅保留编码器部分的单向Transformer,将应答部分抽象成不同的指令,采用孪生网络对指令进行...

Multi-turn Task-oriented Dialogue Technology Based on Unidirectional Transformer and Siamese Network

The existing Recurrent Neural Network(RNN) and Transformer models rely on a large amount of sample data for the modeling of the multi-turn dialogue system,and the accuracy of answering is low.To address the problem,a new modeling method for the task-oriented dialogue system is proposed.Some pre-trained models are introduced for deep encoding of the sentence semantics and the dialog contents.At the same time,the Transformer model is simplified to a unidirectional transformer with only the encoder retained.On this basis,the answering part is abstracted to different commands,which are sorted based on similarity by using the siamese network.The command with the highest similarity is chosen to generate the answer.The experimental results on the MultiWOZ dataset show that compared to LSTM and Transformer-based models,the proposed method has a faster prediction speed,providing better performance on small datasets and equal performance on large datasets.