论文标题
高维贝叶斯对基于注意力的网络的高参数优化以预测材料特性：使用AX和Saasbo对Crabnet的案例研究
High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO
论文作者
论文摘要
昂贵的深度学习模型可以从确定模型体系结构的超参数的优化中受益。我们优化了23个材料信息学模型的超参数，基于组成限制的基于注意力的网络（Crabnet），使用自适应实验（AX）平台中的两个模型超过100个自适应设计迭代。这包括最近开发的贝叶斯优化（BO）算法，稀疏轴对准子空间贝叶斯优化（Saasbo），该子空间在高维优化任务上表现出令人兴奋的性能。使用Saasbo优化Crabnet超参数，我们在材料信息信息基准平台，MATBENCH（〜4.5％的平均绝对误差（MAE）下降〜4.5％）中，在实验带隙回归任务上展示了一个新的最新技术。为每个AX模型描述了自适应设计方案的特征以及特征的重要性。如本工作所示，Saasbo具有改善现有的替代模型的巨大潜力，可以有效地发现高维材料科学搜索空间中的新型，高性能的材料。
Expensive-to-train deep learning models can benefit from an optimization of the hyperparameters that determine the model architecture. We optimize 23 hyperparameters of a materials informatics model, Compositionally-Restricted Attention-Based Network (CrabNet), over 100 adaptive design iterations using two models within the Adaptive Experimentation (Ax) Platform. This includes a recently developed Bayesian optimization (BO) algorithm, sparse axis-aligned subspaces Bayesian optimization (SAASBO), which has shown exciting performance on high-dimensional optimization tasks. Using SAASBO to optimize CrabNet hyperparameters, we demonstrate a new state-of-the-art on the experimental band gap regression task within the materials informatics benchmarking platform, Matbench (~4.5% decrease in mean absolute error (MAE) relative to incumbent). Characteristics of the adaptive design scheme as well as feature importances are described for each of the Ax models. SAASBO has great potential to both improve existing surrogate models, as shown in this work, and in future work, to efficiently discover new, high-performing materials in high-dimensional materials science search spaces.
