论文标题
Attentivedas：通过细心抽样改善神经建筑搜索
AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling
论文作者
论文摘要
神经体系结构搜索（NAS）在设计既准确又高效的最先进模型（SOTA）模型方面表现出了巨大的希望。最近，两阶段的NAS，例如Bignas，分解模型培训和搜索过程，并实现出色的搜索效率和准确性。两阶段NAS需要在培训期间从搜索空间进行抽样，这直接影响了最终搜索模型的准确性。尽管均匀的采样已被广泛用于简单性，但它是模型性能Pareto Front的不可知论，这是搜索过程中的主要重点，因此错过了进一步提高模型准确性的机会。在这项工作中，我们提出了专注于改善采样策略以实现更好性能帕累托的认可。我们还建议算法在训练过程中有效，有效地识别帕累托的网络。如果没有额外的重新训练或后处理，我们可以同时在广泛的拖曳范围内获得大量网络。我们发现的模型家族Attentivenas模型在ImageNet上的TOP-1精度从77.3％到80.7％，并且胜过包括Bignas和曾经是全部网络在内的SOTA模型。我们还仅使用491 mflops实现80.1％的成像网精度。我们的培训代码和预估计的模型可在https://github.com/facebookresearch/attentivenas上找到。
Neural architecture search (NAS) has shown great promise in designing state-of-the-art (SOTA) models that are both accurate and efficient. Recently, two-stage NAS, e.g. BigNAS, decouples the model training and searching process and achieves remarkable search efficiency and accuracy. Two-stage NAS requires sampling from the search space during training, which directly impacts the accuracy of the final searched models. While uniform sampling has been widely used for its simplicity, it is agnostic of the model performance Pareto front, which is the main focus in the search process, and thus, misses opportunities to further improve the model accuracy. In this work, we propose AttentiveNAS that focuses on improving the sampling strategy to achieve better performance Pareto. We also propose algorithms to efficiently and effectively identify the networks on the Pareto during training. Without extra re-training or post-processing, we can simultaneously obtain a large number of networks across a wide range of FLOPs. Our discovered model family, AttentiveNAS models, achieves top-1 accuracy from 77.3% to 80.7% on ImageNet, and outperforms SOTA models, including BigNAS and Once-for-All networks. We also achieve ImageNet accuracy of 80.1% with only 491 MFLOPs. Our training code and pretrained models are available at https://github.com/facebookresearch/AttentiveNAS.
