论文标题
重新访问无培训的NAS指标：一种有效的基于培训的方法
Revisiting Training-free NAS Metrics: An Efficient Training-based Method
论文作者
论文摘要
最近的神经体系结构搜索（NAS）的作品提议的无培训指标对网络进行排名，从而大大降低了NAS的搜索成本。在本文中，我们重新审视了这些无训练的指标，发现：（1）参数数量（\ #param）是最直接的无训练指标，在以前的工作中被忽略了，但令人惊讶的是，（2）最近的无培训指标在很大程度上依赖于\ #param信息\ #param信息\ #param对等级网络的信息。我们的实验表明，当\ #param信息不可用时，最近无培训指标的性能会大大降低。在这些观察结果的推动下，我们认为与\ #param相关的指标是为NAS提供其他信息的。我们提出了一个基于重量训练的指标，该指标与\ #param的相关性较弱，同时比以较低的搜索成本获得了比无训练指标更好的性能。具体而言，在飞镖搜索空间上，我们的方法仅在2.6 GPU小时内直接在ImageNet上完成搜索，并达到24.1 \％/7.1 \％的TOP-1/TOP-5错误率，这在最新的NAS方法之间具有竞争力。代码可在\ url {https://github.com/taoyang1122/revisit_trainingfree_nas}中获得
Recent neural architecture search (NAS) works proposed training-free metrics to rank networks which largely reduced the search cost in NAS. In this paper, we revisit these training-free metrics and find that: (1) the number of parameters (\#Param), which is the most straightforward training-free metric, is overlooked in previous works but is surprisingly effective, (2) recent training-free metrics largely rely on the \#Param information to rank networks. Our experiments show that the performance of recent training-free metrics drops dramatically when the \#Param information is not available. Motivated by these observations, we argue that metrics less correlated with the \#Param are desired to provide additional information for NAS. We propose a light-weight training-based metric which has a weak correlation with the \#Param while achieving better performance than training-free metrics at a lower search cost. Specifically, on DARTS search space, our method completes searching directly on ImageNet in only 2.6 GPU hours and achieves a top-1/top-5 error rate of 24.1\%/7.1\%, which is competitive among state-of-the-art NAS methods. Codes are available at \url{https://github.com/taoyang1122/Revisit_TrainingFree_NAS}
