论文标题
神经网络中的注意机制：它到达的地方和去向
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes
论文作者
论文摘要
很久以前，在机器学习文献中，引入了将人类视觉系统启发的机制纳入神经网络的想法。这个想法被命名为注意机制，并且已经经历了漫长的发展时期。如今，许多作品都在各种任务中致力于这一想法。最近已经证明了出色的表现。本文的目的是提供早期工作的概述，以搜索使用神经网络实施注意力思想的方法，直到最近的趋势为止。这篇评论强调了在这一进度中有关不同任务的重要里程碑。通过这种方式，这项研究旨在为研究人员提供路线图，以探索当前的发展，并为不受关注的新方法而受到启发。
A long time ago in the machine learning literature, the idea of incorporating a mechanism inspired by the human visual system into neural networks was introduced. This idea is named the attention mechanism, and it has gone through a long development period. Today, many works have been devoted to this idea in a variety of tasks. Remarkable performance has recently been demonstrated. The goal of this paper is to provide an overview from the early work on searching for ways to implement attention idea with neural networks until the recent trends. This review emphasizes the important milestones during this progress regarding different tasks. By this way, this study aims to provide a road map for researchers to explore the current development and get inspired for novel approaches beyond the attention.
