论文标题
矢量符号的残留和注意力结构
Residual and Attentional Architectures for Vector-Symbols
论文作者
论文摘要
矢量符号体系结构（VSA）提供了高度灵活并具有独特优势的计算方法。 VSA中的概念由“符号”，值的长量向量表示，这些值利用高维空间的属性来表示和操纵信息。在这项新作品中，我们将在傅立叶全息降低表示（FHRR）VSA的框架内提供的操作的效率与深层网络的力量构建新型VSA基于VSA的残差和基于注意力的神经网络体系结构。使用注意力FHRR体系结构，我们证明了相同的网络体系结构可以通过将不同的信息编码到网络的输入中，类似于感知者模型，从而解决了来自不同域（图像分类和分子毒性预测）的问题。这证明了VSA的新应用以及在神经形态硬件上实施最新神经模型的潜在途径。
Vector-symbolic architectures (VSAs) provide methods for computing which are highly flexible and carry unique advantages. Concepts in VSAs are represented by 'symbols,' long vectors of values which utilize properties of high-dimensional spaces to represent and manipulate information. In this new work, we combine efficiency of the operations provided within the framework of the Fourier Holographic Reduced Representation (FHRR) VSA with the power of deep networks to construct novel VSA based residual and attention-based neural network architectures. Using an attentional FHRR architecture, we demonstrate that the same network architecture can address problems from different domains (image classification and molecular toxicity prediction) by encoding different information into the network's inputs, similar to the Perceiver model. This demonstrates a novel application of VSAs and a potential path to implementing state-of-the-art neural models on neuromorphic hardware.
