论文标题
用于回归的框架和基准,用于深入批处理积极学习
A Framework and Benchmark for Deep Batch Active Learning for Regression
论文作者
论文摘要
获得用于监督学习的标签可能很昂贵。为了提高神经网络回归的样本效率,我们研究了活跃的学习方法,这些方法可以适应地选择未标记的数据进行标记。我们提出了一个框架,用于从(与网络相关的)基础内核,内核变换和选择方法中构造此类方法。我们的框架涵盖了许多基于神经网络的高斯过程近似以及非bayesian方法的现有贝叶斯方法。此外,我们建议用草图的有限宽度神经切线内核代替常用的最后层特征,并将它们与一种新型的聚类方法相结合。为了评估不同的方法,我们引入了一个由15个大型表的回归数据集组成的开源基准。我们所提出的方法的表现优于我们的基准测试上的最新方法,缩放到大数据集,并在不调整网络体系结构或培训代码的情况下开箱即用。我们提供开源代码,其中包括所有内核,内核转换和选择方法的有效实现,并可用于复制我们的结果。
The acquisition of labels for supervised learning can be expensive. To improve the sample efficiency of neural network regression, we study active learning methods that adaptively select batches of unlabeled data for labeling. We present a framework for constructing such methods out of (network-dependent) base kernels, kernel transformations, and selection methods. Our framework encompasses many existing Bayesian methods based on Gaussian process approximations of neural networks as well as non-Bayesian methods. Additionally, we propose to replace the commonly used last-layer features with sketched finite-width neural tangent kernels and to combine them with a novel clustering method. To evaluate different methods, we introduce an open-source benchmark consisting of 15 large tabular regression data sets. Our proposed method outperforms the state-of-the-art on our benchmark, scales to large data sets, and works out-of-the-box without adjusting the network architecture or training code. We provide open-source code that includes efficient implementations of all kernels, kernel transformations, and selection methods, and can be used for reproducing our results.