论文标题
回声状态网络的设计和应用的综述
A Review of Designs and Applications of Echo State Networks
论文作者
论文摘要
经常性的神经网络(RNN)表现出了它们在序列任务上的出色能力,并在工业,医学,经济和语言等广泛应用中实现了最新的能力。回声状态网络(ESN)是简单的RNN类型,在过去十年中已成为基于梯度下降训练的RNN的一种替代方案。 ESN具有强大的理论基础,是实用的,在概念上很简单,易于实施。它避免在梯度下降方法中避免不结合和计算昂贵。自2002年提出ESN以来,现有的大量作品促进了ESN的进步,而最近引入的深层ESN模型为团结深度学习和ESN的优点开辟了道路。此外,在某些应用程序中,ESN与其他机器学习模型的组合也表现出了过分的基线。但是,ESN的明显简单性有时可以具有欺骗性,并且成功地应用ESN需要一些经验。因此,在本文中,我们将基于ESN的方法分类为基本的ESN,DEEPESN和组合,然后从理论研究,网络设计和特定应用的角度分析它们。最后,我们通过总结开放问题并提出可能的未来工作来讨论ESN的挑战和机遇。
Recurrent Neural Networks (RNNs) have demonstrated their outstanding ability in sequence tasks and have achieved state-of-the-art in wide range of applications, such as industrial, medical, economic and linguistic. Echo State Network (ESN) is simple type of RNNs and has emerged in the last decade as an alternative to gradient descent training based RNNs. ESN, with a strong theoretical ground, is practical, conceptually simple, easy to implement. It avoids non-converging and computationally expensive in the gradient descent methods. Since ESN was put forward in 2002, abundant existing works have promoted the progress of ESN, and the recently introduced Deep ESN model opened the way to uniting the merits of deep learning and ESNs. Besides, the combinations of ESNs with other machine learning models have also overperformed baselines in some applications. However, the apparent simplicity of ESNs can sometimes be deceptive and successfully applying ESNs needs some experience. Thus, in this paper, we categorize the ESN-based methods to basic ESNs, DeepESNs and combinations, then analyze them from the perspective of theoretical studies, network designs and specific applications. Finally, we discuss the challenges and opportunities of ESNs by summarizing the open questions and proposing possible future works.