论文标题
PACOH：贝叶斯最佳的元学习和PAC保证
PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees
论文作者
论文摘要
元学习可以成功地从数据中获得有用的归纳偏见。然而，它对看不见的学习任务的概括属性知之甚少。特别是如果元训练任务的数量很少，这引起了人们对过度拟合的担忧。我们使用Pac-Bayesian框架提供了理论分析，并为元学习提供了新的概括界。使用这些界限，我们开发了具有性能保证和原则性元级正则化的一类PAC-optimal元学习算法。与以前的pac-bayesian元学习者不同，我们的方法会导致标准的随机优化问题，该问题可以有效地解决并很好地缩放。当我们以高斯过程和贝叶斯神经网络作为基础学习者实例化我们的PAC-optimal高足体（PACOH）时，所产生的方法在预测准确性和不确定性估计的质量方面都会产生最先进的表现。由于他们对不确定性的原则处理，我们的元学习者也可以成功用于顺序决策问题。
Meta-learning can successfully acquire useful inductive biases from data. Yet, its generalization properties to unseen learning tasks are poorly understood. Particularly if the number of meta-training tasks is small, this raises concerns about overfitting. We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning. Using these bounds, we develop a class of PAC-optimal meta-learning algorithms with performance guarantees and a principled meta-level regularization. Unlike previous PAC-Bayesian meta-learners, our method results in a standard stochastic optimization problem which can be solved efficiently and scales well. When instantiating our PAC-optimal hyper-posterior (PACOH) with Gaussian processes and Bayesian Neural Networks as base learners, the resulting methods yield state-of-the-art performance, both in terms of predictive accuracy and the quality of uncertainty estimates. Thanks to their principled treatment of uncertainty, our meta-learners can also be successfully employed for sequential decision problems.
