论文标题
线性RNN可证明学习线性动态系统
Linear RNNs Provably Learn Linear Dynamic Systems
论文作者
论文摘要
我们研究具有梯度下降的线性复发神经网络的学习能力。我们证明了在线性RNN上的第一个理论保证，可以使用任何大型损耗函数学习任何稳定的线性动态系统。 For an arbitrary stable linear system with a parameter $ρ_C$ related to the transition matrix $C$, we show that despite the non-convexity of the parameter optimization loss if the width of the RNN is large enough (and the required width in hidden layers does not rely on the length of the input sequence), a linear RNN can provably learn any stable linear dynamic system with the sample and time complexity polynomial in $ \ frac {1} {1-ρ_C} $。我们的结果为学习线性RNN提供了第一个理论保证，并演示了经常性结构如何帮助学习动态系统。
We study the learning ability of linear recurrent neural networks with Gradient Descent. We prove the first theoretical guarantee on linear RNNs to learn any stable linear dynamic system using any a large type of loss functions. For an arbitrary stable linear system with a parameter $ρ_C$ related to the transition matrix $C$, we show that despite the non-convexity of the parameter optimization loss if the width of the RNN is large enough (and the required width in hidden layers does not rely on the length of the input sequence), a linear RNN can provably learn any stable linear dynamic system with the sample and time complexity polynomial in $\frac{1}{1-ρ_C}$. Our results provide the first theoretical guarantee to learn a linear RNN and demonstrate how can the recurrent structure help to learn a dynamic system.
