论文标题
摊销条件归一化的最大可能性：可靠的分布不确定性估计
Amortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation
论文作者
论文摘要
尽管深层神经网络为一系列具有挑战性的任务提供了良好的性能，但校准和不确定性估计仍然是主要的挑战，尤其是在分销变化下。在本文中，我们提出了摊销的条件归一化最大似然（ACNML）方法，作为一种可扩展的通用方法，用于与深网的不确定性估计，校准和分布外的鲁棒性。我们的算法建立在条件归一化的最大似然（CNML）编码方案上，该方案具有最小值的最佳属性，根据最小描述长度原理，但在计算上非常棘手，可以准确地评估除最简单的模型类外的所有内容。我们建议使用近似贝叶斯推理技术来产生与CNML分布的典型近似。我们的方法可以与任何近似推理算法结合使用，该算法可提供与模型参数相比的可拖动后密度。我们证明，ACNML与许多先前的技术相比，在分布外输入的校准方面进行了不确定性估计。
While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges, especially under distribution shift. In this paper, we propose the amortized conditional normalized maximum likelihood (ACNML) method as a scalable general-purpose approach for uncertainty estimation, calibration, and out-of-distribution robustness with deep networks. Our algorithm builds on the conditional normalized maximum likelihood (CNML) coding scheme, which has minimax optimal properties according to the minimum description length principle, but is computationally intractable to evaluate exactly for all but the simplest of model classes. We propose to use approximate Bayesian inference technqiues to produce a tractable approximation to the CNML distribution. Our approach can be combined with any approximate inference algorithm that provides tractable posterior densities over model parameters. We demonstrate that ACNML compares favorably to a number of prior techniques for uncertainty estimation in terms of calibration on out-of-distribution inputs.
