论文标题
关于变异推理和自动求解记忆之间的关系
On the Relationship Between Variational Inference and Auto-Associative Memory
论文作者
论文摘要
在本文中，我们提出了自动缔合性记忆的各种推理表述，使我们能够将感知推理和记忆检索结合到相同的数学框架中。在此公式中，对潜在表示的先验概率分布依赖于内存，从而将推理过程提高到先前存储的表示。然后，我们研究如何在此框架中应用不同的变异推理方法。我们比较依赖于摊销推理的方法，例如变异自动编码器和依赖迭代推理的方法，例如预测性编码，并建议将两种方法组合起来设计新的自动求解记忆模型。我们评估了CIFAR10和CLEVR图像数据集上获得的算法，并将它们与其他关联内存模型（例如Hopfield Network，端到端存储网络和神经图灵机）进行比较。
In this article, we propose a variational inference formulation of auto-associative memories, allowing us to combine perceptual inference and memory retrieval into the same mathematical framework. In this formulation, the prior probability distribution onto latent representations is made memory dependent, thus pulling the inference process towards previously stored representations. We then study how different neural network approaches to variational inference can be applied in this framework. We compare methods relying on amortized inference such as Variational Auto Encoders and methods relying on iterative inference such as Predictive Coding and suggest combining both approaches to design new auto-associative memory models. We evaluate the obtained algorithms on the CIFAR10 and CLEVR image datasets and compare them with other associative memory models such as Hopfield Networks, End-to-End Memory Networks and Neural Turing Machines.
