论文标题
了解基于梯度的元学习中的良性过度拟合
Understanding Benign Overfitting in Gradient-Based Meta Learning
论文作者
论文摘要
元学习在有限的监督数据中表现出了几次学习的巨大成功。在这些设置中，元模型通常被过度参数化。尽管常规的统计学习理论表明，过度参数化的模型倾向于过度合适，但经验证据表明，过度参数化的元学习方法仍然很好地工作 - 这种现象通常称为“良性过度拟合”。为了了解这种现象，我们将重点放在具有挑战性的双层结构的元学习设置上，我们将基于梯度的元学习称为，并在过度参数化的元线性回归模型下分析其泛化性能。尽管我们的分析使用了相对可牵引的线性模型，但我们的理论有助于理解基于梯度的元学习任务中数据异质性，模型适应性和良性过量之间的微妙相互作用。我们通过数值模拟证实了我们的理论主张。
Meta learning has demonstrated tremendous success in few-shot learning with limited supervised data. In those settings, the meta model is usually overparameterized. While the conventional statistical learning theory suggests that overparameterized models tend to overfit, empirical evidence reveals that overparameterized meta learning methods still work well -- a phenomenon often called "benign overfitting." To understand this phenomenon, we focus on the meta learning settings with a challenging bilevel structure that we term the gradient-based meta learning, and analyze its generalization performance under an overparameterized meta linear regression model. While our analysis uses the relatively tractable linear models, our theory contributes to understanding the delicate interplay among data heterogeneity, model adaptation and benign overfitting in gradient-based meta learning tasks. We corroborate our theoretical claims through numerical simulations.
