论文标题
神经传送
Neural Teleportation
论文作者
论文摘要
在本文中，我们探讨了一个称为神经传送的过程，这是将箭量表示理论应用于神经网络的数学结果。神经传送 将网络“传送”到重量空间中的新位置并保留其功能。这种现象直接来自应用于神经网络的表示理论的定义，事实证明这是一个非常简单的操作，具有出色的特性。我们阐明了神经传送对损失景观的惊人和反直觉后果。特别是，我们表明传送范围可用于探索损失水平曲线，它改变了当地的损失格局，在学习过程中的任何时候都会随时增加全球最小值并增强后置梯度。我们的结果可以在此处可用的代码中复制：https：//github.com/vitalab/neuralteleportation
In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation "teleports" a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process. Our results can be reproduced with the code available here: https://github.com/vitalab/neuralteleportation
