Recurrent neural networks and prior knowledge for sequence processing: a constrained nondeterministic approach

The paper focuses on methods for injecting prior knowledge into adaptive recurrent networks for sequence processing. In order to increase the flexibility needed for specifying partially known rules, a nondeterministic approach for modelling domain knowledge is proposed. The algorithms presented in the paper allow time-warping nondeterministic automata to be mapped into recurrent architectures with first-order connections. These kinds of automata are suitable for modeling temporal scale distortions in data such as acoustic sequences occurring in problems of speech recognition. The algorithms output a recurrent architecture and a feasible region in the connection weight space. It is demonstrated that, as long as the weights are constrained into the feasible region, the nondeterministic rules introduced using prior knowledge are not destroyed by learning. The paper focuses primarily on architectural issues, but the proposed method allows the connection weights to be subsequently tuned to adapt the behavior of the network to data.