论文标题
通过神经特征对齐的隐式正则化
Implicit Regularization via Neural Feature Alignment
论文作者
论文摘要
我们从几何观点中深入学习中的隐式正规化问题。我们强调了Jacot等人沿少数与任务相关的方向引入的神经切线特征的动态比对引起的正则化效应。这可以解释为特征选择和压缩的组合机制。通过推断线性模型的Rademacher复杂性界限的新分析，我们通过沿优化路径的切线核类别的序列来激励和研究一种启发式复杂性度量，该测量值量捕获了这种现象。
We approach the problem of implicit regularization in deep learning from a geometrical viewpoint. We highlight a regularization effect induced by a dynamical alignment of the neural tangent features introduced by Jacot et al, along a small number of task-relevant directions. This can be interpreted as a combined mechanism of feature selection and compression. By extrapolating a new analysis of Rademacher complexity bounds for linear models, we motivate and study a heuristic complexity measure that captures this phenomenon, in terms of sequences of tangent kernel classes along optimization paths.
