论文标题
通过基于任务相似性的元学习来加快多目标超参数优化，以供树结构化parzen估算器
Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator
论文作者
论文摘要
超参数优化（HPO）是改善深度学习（DL）性能的重要步骤。从业者通常面临多个标准之间的权衡，例如准确性和延迟。考虑到DL的高计算需求以及对高效HPO的需求不断增长，多目标（MO）优化的加速度变得越来越重要。尽管在HPO的元学习方面进行了重要的工作，但现有方法对Mo Tree结构的Parzen估计量（MO-TPE）不适用，这是一种简单而功能强大的MO-HPO算法。在本文中，我们使用由任务之间顶级域的重叠定义的任务相似性将TPE的采集功能扩展到元学习设置。我们还理论上分析并解决了任务相似性的局限性。在实验中，我们证明我们的方法在表格HPO基准测试基准上加快了MO-TPE，并达到了最先进的性能。我们的方法还通过赢得“变压器的多目标超参数优化”的Automl 2022竞争来在外部进行验证。
Hyperparameter optimization (HPO) is a vital step in improving performance in deep learning (DL). Practitioners are often faced with the trade-off between multiple criteria, such as accuracy and latency. Given the high computational needs of DL and the growing demand for efficient HPO, the acceleration of multi-objective (MO) optimization becomes ever more important. Despite the significant body of work on meta-learning for HPO, existing methods are inapplicable to MO tree-structured Parzen estimator (MO-TPE), a simple yet powerful MO-HPO algorithm. In this paper, we extend TPE's acquisition function to the meta-learning setting using a task similarity defined by the overlap of top domains between tasks. We also theoretically analyze and address the limitations of our task similarity. In the experiments, we demonstrate that our method speeds up MO-TPE on tabular HPO benchmarks and attains state-of-the-art performance. Our method was also validated externally by winning the AutoML 2022 competition on "Multiobjective Hyperparameter Optimization for Transformers".
