论文标题
校准的合奏可以减轻分配转移的准确性权衡
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift
论文作者
论文摘要
我们经常看到在强大的机器学习中不良的权衡，在这种学习中，分布（OOD）的精度与分布式（ID）的准确性（ID）准确性相矛盾：通过删除虚假功能（例如，与通过ERM受过的标准分类训练的标准分类器相比，诸如删除伪造功能的专用技术）获得的强大分类器通常具有更好的OOD，但ID的准确性更高。在本文中，我们发现，ID校准的合奏仅在校准ID数据后简单地将标准和健壮的模型进行了整合 - 在ID和OOD准确性上都优于先前的最新先进（基于自我训练）。在11个自然分配偏移数据集中，ID校准的合奏获得了两全其美的最佳：强大的ID准确性和OOD精度。我们在风格化的设置中分析了此方法，并确定了两个重要条件以使合奏执行ID和OOD都很好：（1）我们需要校准标准模型和健壮的模型（在ID数据上，因为OOD数据不可用），（2）OOD没有反相关的虚假特征。
We often see undesirable tradeoffs in robust machine learning where out-of-distribution (OOD) accuracy is at odds with in-distribution (ID) accuracy: a robust classifier obtained via specialized techniques such as removing spurious features often has better OOD but worse ID accuracy compared to a standard classifier trained via ERM. In this paper, we find that ID-calibrated ensembles -- where we simply ensemble the standard and robust models after calibrating on only ID data -- outperforms prior state-of-the-art (based on self-training) on both ID and OOD accuracy. On eleven natural distribution shift datasets, ID-calibrated ensembles obtain the best of both worlds: strong ID accuracy and OOD accuracy. We analyze this method in stylized settings, and identify two important conditions for ensembles to perform well both ID and OOD: (1) we need to calibrate the standard and robust models (on ID data, because OOD data is unavailable), (2) OOD has no anticorrelated spurious features.
