深层神经网络中间层可见化建模

深层神经网络的中间层是隐含的、未知的,这使得深层网络的学习过程不可追踪,学习结果无法解释,在一 定程度上制约了深度学习的发展.本文通过引入先验知识使深层网络的中间层具有明确的含义与显性的影响 关系,即中间层可见化,从而部分人工干预深层网络的内部结构,约束网络学习的方向.基于深层堆叠网 络 (Deep stacking network, DSN),提出两种中间层部分可见的深层神经网络:输入层部分可见的深层堆叠网络(Input-layer visible DSN, IVDSN)和隐含层部分可见的深层堆叠网络(Hidden-layer visible DSN, HVDSN),部分可见是为了保留对未知信息的提取能力和一定的容错能力.以基于文本的言语情 感计算为例测试所提网络的有效性,结果表明先验知识的引入有助于提升深层神经网络的 性能;所提两种网络均可实现中间层的部分可见化,其中HVDSN结构更精简,性能也更优.

Deep Neural Networks with Visible Intermediate Layers

The hidden nature of intermediate layers in deep neural networks makes the learning process hard to track and the learned results difficult to explain, which restricts the development of deep networks to some extent. This work focuses on making these intermediate layers visible through prior knowledge, which means giving the intermediate layers definite meanings and explicit interrelationship, in the hope to supervise the learning process of deep networks and guide the learning direction. On the basis of deep stacking network (DSN), we propose two networks in which the intermediate layers are partially visible: the input-layer visible deep stacking network (IVDSN) and the hidden-layer visible deep stacking network (HVDSN). To be partially but not fully visible is to leave room for the unknown and the error. With the application of the text-based detection of speech emotion, the performance of the proposed networks is tested. The results validate that the transparency of intermediate layers is beneficial to improve the performance of deep neural networks. Between the two proposed networks, the HVDSN has a simpler structure and a better performance.