论文标题
元学习估计误差的理论界限
Theoretical bounds on estimation error for meta-learning
论文作者
论文摘要
传统上，机器学习模型是在训练和测试分布完全匹配的假设下开发的。但是，在几次学习和相关问题中，最近的成功令人鼓舞，表明这些模型可以适应更现实的设置，在这些设置中，火车和测试分布有所不同。不幸的是，对这些算法的理论支持严格有限，对这些问题的困难知之甚少。在这项工作中，我们为算法的最小收敛速率提供了新的信息理论较低限制，这些算法受到来自多个来源的数据并对新数据进行了测试的培训。我们的界限直观地取决于数据源之间共享的信息，并表征了在此环境中学习任意算法的难度。我们在分层贝叶斯模型的元学习模型上演示了这些界限，并通过最大A-Posteriori推断计算上和下限的参数估计。
Machine learning models have traditionally been developed under the assumption that the training and test distributions match exactly. However, recent success in few-shot learning and related problems are encouraging signs that these models can be adapted to more realistic settings where train and test distributions differ. Unfortunately, there is severely limited theoretical support for these algorithms and little is known about the difficulty of these problems. In this work, we provide novel information-theoretic lower-bounds on minimax rates of convergence for algorithms that are trained on data from multiple sources and tested on novel data. Our bounds depend intuitively on the information shared between sources of data, and characterize the difficulty of learning in this setting for arbitrary algorithms. We demonstrate these bounds on a hierarchical Bayesian model of meta-learning, computing both upper and lower bounds on parameter estimation via maximum-a-posteriori inference.
