论文标题
提高置信度估计的可靠性
Improving the Reliability for Confidence Estimation
论文作者
论文摘要
信心估计是一项旨在评估模型在部署过程中预测输出的可信度的任务，最近由于其对安全部署的重要性而引起了很多研究的关注。先前的工作概述了两个重要素质，可靠的置信度估计模型应具有，即在标签不平衡下表现良好的能力以及处理各种分布数据输入的能力。在这项工作中，我们提出了一个元学习框架，可以在置信度估计模型中同时提高这两种质量。具体来说，我们首先构建虚拟培训和测试集，并在它们之间有一些有意设计的分配差异。然后，我们的框架使用构造的集合通过虚拟培训和测试方案来训练置信度估计模型，从而导致其学习知识，以推广到各种分布。我们显示了框架对单眼深度估计和图像分类的有效性。
Confidence estimation, a task that aims to evaluate the trustworthiness of the model's prediction output during deployment, has received lots of research attention recently, due to its importance for the safe deployment of deep models. Previous works have outlined two important qualities that a reliable confidence estimation model should possess, i.e., the ability to perform well under label imbalance and the ability to handle various out-of-distribution data inputs. In this work, we propose a meta-learning framework that can simultaneously improve upon both qualities in a confidence estimation model. Specifically, we first construct virtual training and testing sets with some intentionally designed distribution differences between them. Our framework then uses the constructed sets to train the confidence estimation model through a virtual training and testing scheme leading it to learn knowledge that generalizes to diverse distributions. We show the effectiveness of our framework on both monocular depth estimation and image classification.
