Designing ladder and laguerre orthogonal recurrent neural network architectures inspired by discrete-time dynamical systems

Abstract

A method and system for making a prediction with a recurrent neural network (RNN) of a discrete-time linear time-invariant system can receive, as an input into a recurrent neural network cell, a memory flow, an external input, a state flow and a hidden output flow. The RNN can compute an updated state flow by a linear combination of the external input and the memory flow constrained by a first matrix and a second matrix, wherein the first matrix and the second matrix correspond to a Laguerre orthogonal basis. The RNN can further compute, from the updated state flow using a coordinate matrix, an updated memory flow. The RNN can then apply an activation function on the hidden output flow to produce an updated hidden output flow and an output flow from a weighted sum of the hidden output flow, the updated state flow and the updated memory flow.