论文标题
来自嘈杂的边际似然估计的贝叶斯优化超参数
Bayesian Optimization of Hyperparameters from Noisy Marginal Likelihood Estimates
论文作者
论文摘要
贝叶斯模型通常涉及通过最大化边际可能性确定的一小部分超参数。贝叶斯优化是一种流行的迭代方法，其中基础函数的高斯过程被新功能评估依次更新。采集策略使用此后验分布来决定在何处放置下一个功能评估。我们为用户控制计算工作的情况以及函数评估的精确度提出了一个新颖的贝叶斯优化框架。这是计量经济学中的常见情况，即马尔可夫链蒙特卡洛（MCMC）通常计算出边际可能性或重要性采样方法，而边际可能性估计仪的精确度由样品数量确定。新的采集策略使优化器可以选择通过廉价的嘈杂评估来探索该功能，因此可以更快地找到最佳。该方法应用于在US宏观经济时间序列数据中估算两个流行模型中的先前的超参数：稳态贝叶斯矢量自回归（BVAR）和具有随机波动性的时变参数BVAR。所提出的方法显示出比传统的贝叶斯优化或网格搜索要快得多。
Bayesian models often involve a small set of hyperparameters determined by maximizing the marginal likelihood. Bayesian optimization is a popular iterative method where a Gaussian process posterior of the underlying function is sequentially updated by new function evaluations. An acquisition strategy uses this posterior distribution to decide where to place the next function evaluation. We propose a novel Bayesian optimization framework for situations where the user controls the computational effort, and therefore the precision of the function evaluations. This is a common situation in econometrics where the marginal likelihood is often computed by Markov chain Monte Carlo (MCMC) or importance sampling methods, with the precision of the marginal likelihood estimator determined by the number of samples. The new acquisition strategy gives the optimizer the option to explore the function with cheap noisy evaluations and therefore find the optimum faster. The method is applied to estimating the prior hyperparameters in two popular models on US macroeconomic time series data: the steady-state Bayesian vector autoregressive (BVAR) and the time-varying parameter BVAR with stochastic volatility. The proposed method is shown to find the optimum much quicker than traditional Bayesian optimization or grid search.
