论文标题
通过可行的指标重建积极学习的信任
Rebuilding Trust in Active Learning with Actionable Metrics
论文作者
论文摘要
主动学习（AL）是研究的活跃领域，但尽管需要紧迫，但在行业中很少使用。这部分是由于目标的不对准，而研究努力在选定的数据集中获得最佳结果，但该行业希望确保积极学习的表现将始终如一，至少比随机标签更好。积极学习的一次性本质使得了解如何进行策略选择以及促进性能差的是什么（缺乏探索，选择难以分类的样本的选择，...）。 为了帮助重建工业从业者在积极学习方面的信任，我们提出了各种可行的指标。通过对参考数据集（例如Cifar100，Fashion-Mnist和20newsgroups）进行的广泛实验，我们表明这些指标可以为实践者可以利用的策略带来解释性。
Active Learning (AL) is an active domain of research, but is seldom used in the industry despite the pressing needs. This is in part due to a misalignment of objectives, while research strives at getting the best results on selected datasets, the industry wants guarantees that Active Learning will perform consistently and at least better than random labeling. The very one-off nature of Active Learning makes it crucial to understand how strategy selection can be carried out and what drives poor performance (lack of exploration, selection of samples that are too hard to classify, ...). To help rebuild trust of industrial practitioners in Active Learning, we present various actionable metrics. Through extensive experiments on reference datasets such as CIFAR100, Fashion-MNIST, and 20Newsgroups, we show that those metrics brings interpretability to AL strategies that can be leveraged by the practitioner.
