论文标题
贝叶斯神经网络的模型架构改编
Model Architecture Adaption for Bayesian Neural Networks
论文作者
论文摘要
贝叶斯神经网络（BNNS）提供了一个数学上的框架，以量化模型预测的不确定性，但在培训和推理中都具有过度的计算成本。在这项工作中，我们展示了一种新颖的网络体系结构搜索（NAS），该搜索优化了BNN的精度和不确定性，同时减少了推理潜伏期。与仅用于分布的可能性优化的规范NA不同，建议的方案搜索了使用分布式和分布外数据的不确定性性能。我们的方法能够搜索网络中贝叶斯层的正确放置。在我们的实验中，与最先进的（深度集合）相比，搜索模型显示出可比的不确定性量化能力和准确性。此外，与许多流行的BNN基线相比，搜索型号仅使用运行时的一小部分，与McDropout和Deep Ensemble相比，CIFAR10数据集的推理运行时成本分别$ 2.98 \ times $ $和$ 2.92 \ times $。
Bayesian Neural Networks (BNNs) offer a mathematically grounded framework to quantify the uncertainty of model predictions but come with a prohibitive computation cost for both training and inference. In this work, we show a novel network architecture search (NAS) that optimizes BNNs for both accuracy and uncertainty while having a reduced inference latency. Different from canonical NAS that optimizes solely for in-distribution likelihood, the proposed scheme searches for the uncertainty performance using both in- and out-of-distribution data. Our method is able to search for the correct placement of Bayesian layer(s) in a network. In our experiments, the searched models show comparable uncertainty quantification ability and accuracy compared to the state-of-the-art (deep ensemble). In addition, the searched models use only a fraction of the runtime compared to many popular BNN baselines, reducing the inference runtime cost by $2.98 \times$ and $2.92 \times$ respectively on the CIFAR10 dataset when compared to MCDropout and deep ensemble.
