论文标题
具有随机特征的高维学习的通用法律
Universality Laws for High-Dimensional Learning with Random Features
论文作者
论文摘要
我们证明了具有随机特征学习的通用定理。我们的结果表明，就训练和概括错误而言，具有非线性激活函数的随机特征模型在渐近上等同于具有匹配协方差矩阵的替代线性高斯模型。这解决了一个所谓的高斯等效性猜想，基于最近的几篇论文发展其结果。我们证明普遍性定理的方法是基于古典Lindeberg方法的。证明的主要成分包括针对与训练过程相关的优化问题和通过Stein方法获得的中心限制定理的遗留分析，以实现弱相关的随机变量。
We prove a universality theorem for learning with random features. Our result shows that, in terms of training and generalization errors, a random feature model with a nonlinear activation function is asymptotically equivalent to a surrogate linear Gaussian model with a matching covariance matrix. This settles a so-called Gaussian equivalence conjecture based on which several recent papers develop their results. Our method for proving the universality theorem builds on the classical Lindeberg approach. Major ingredients of the proof include a leave-one-out analysis for the optimization problem associated with the training process and a central limit theorem, obtained via Stein's method, for weakly correlated random variables.
