论文标题
Pasha：有效的HPO和NAS具有渐进资源分配
PASHA: Efficient HPO and NAS with Progressive Resource Allocation
论文作者
论文摘要
超参数优化（HPO）和神经体系结构搜索（NAS）是获得一流的机器学习模型的选择，但实际上，它们的运行可能会很高。当在大型数据集上培训模型时，即使采用了有效的多忠诚方法，对从业者进行HPO或NAS的调整迅速昂贵。我们提出了一种方法，以应对在具有有限计算资源的大型数据集上培训的调整机器学习模型的挑战。我们的方法名为Pasha，扩展了ASHA，并能够根据需要动态分配最大资源为调整过程。实验比较表明，Pasha识别出良好的超参数配置和体系结构，而消耗的计算资源明显少于ASHA。
Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in practice they can be costly to run. When models are trained on large datasets, tuning them with HPO or NAS rapidly becomes prohibitively expensive for practitioners, even when efficient multi-fidelity methods are employed. We propose an approach to tackle the challenge of tuning machine learning models trained on large datasets with limited computational resources. Our approach, named PASHA, extends ASHA and is able to dynamically allocate maximum resources for the tuning procedure depending on the need. The experimental comparison shows that PASHA identifies well-performing hyperparameter configurations and architectures while consuming significantly fewer computational resources than ASHA.
