论文标题
光谱混合物内核的近似推断
Approximate Inference for Spectral Mixture Kernel
论文作者
论文摘要
光谱混合物（SM）内核是一种灵活的内核，用于对任何固定协方差函数进行建模。尽管它在建模数据中很有用，但是SM内核的学习通常很困难，因为优化SM内核的大量参数通常会导致过度拟合，尤其是当使用基于梯度的优化时。此外，还需要更长的培训时间。为了改善培训，我们提出了对SM内核的大约贝叶斯推断。具体而言，我们采用光谱点的变异分布来近似具有随机傅立叶特征的SM内核。我们通过将基于抽样的变异推断应用于根据近似内核构建的衍生证据下限（ELBO）估计量来优化变分参数。为了改善推论，我们进一步提出了两种其他策略：（1）光谱点的抽样策略可靠地估算Elbo估计值，从而估算其相关梯度，以及（2）近似自然梯度，以加速参数的收敛性。所提出的推理与两种策略结合加速了参数的收敛性，并导致更好的最佳参数。
A spectral mixture (SM) kernel is a flexible kernel used to model any stationary covariance function. Although it is useful in modeling data, the learning of the SM kernel is generally difficult because optimizing a large number of parameters for the SM kernel typically induces an over-fitting, particularly when a gradient-based optimization is used. Also, a longer training time is required. To improve the training, we propose an approximate Bayesian inference for the SM kernel. Specifically, we employ the variational distribution of the spectral points to approximate SM kernel with a random Fourier feature. We optimize the variational parameters by applying a sampling-based variational inference to the derived evidence lower bound (ELBO) estimator constructed from the approximate kernel. To improve the inference, we further propose two additional strategies: (1) a sampling strategy of spectral points to estimate the ELBO estimator reliably and thus its associated gradient, and (2) an approximate natural gradient to accelerate the convergence of the parameters. The proposed inference combined with two strategies accelerates the convergence of the parameters and leads to better optimal parameters.
