A hybrid language model based on a recurrent neural network and probabilistic topic modeling |
| |
Authors: | M. S. Kudinov A. A. Romanenko |
| |
Affiliation: | 1.Federal Research Center Computer Science and Control,Russian Academy of Sciences,Moscow,Russia;2.Moscow Institute of Physics and Technology (State University),Dolgoprudnyi,Russia |
| |
Abstract: | A language model based on features extracted from a recurrent neural network language model and semantic embedding of the left context of the current word based on probabilistic semantic analysis (PLSA) is developed. To calculate such embedding, the context is considered as a document. The effect of vanishing gradients in a recurrent neural network is reduced by this method. The experiment has shown that adding topic-based features reduces perplexity by 10%. |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |