论文标题
梯度指导的重要性抽样用于学习二进制能量的模型
Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models
论文作者
论文摘要
众所周知，基于学习能量的模型（EBM）很难，尤其是在无法直接应用基于梯度的学习策略的离散数据上。尽管比率匹配是学习离散EBM的合理方法，但它具有昂贵的计算和过度的内存要求，从而导致在高维数据上学习EBM的困难。在这些局限性的推动下，在这项研究中，我们提出与梯度引导的重要性采样（RMWGGIS）匹配的比率。特别是，我们使用能量函数W.R.T.的梯度离散的数据空间近似构建可证明的最佳提案分布，随后通过重要性采样来使用该空间来有效估计原始比率匹配目标。我们对合成离散数据，图生成和训练模型进行密度建模进行实验，以评估我们的建议方法。实验结果表明，我们的方法可以显着缓解比率匹配的局限性，在实践中更有效，并扩展到高维问题。我们的实施可从https://github.com/divelab/rmwggis获得。
Learning energy-based models (EBMs) is known to be difficult especially on discrete data where gradient-based learning strategies cannot be applied directly. Although ratio matching is a sound method to learn discrete EBMs, it suffers from expensive computation and excessive memory requirements, thereby resulting in difficulties in learning EBMs on high-dimensional data. Motivated by these limitations, in this study, we propose ratio matching with gradient-guided importance sampling (RMwGGIS). Particularly, we use the gradient of the energy function w.r.t. the discrete data space to approximately construct the provably optimal proposal distribution, which is subsequently used by importance sampling to efficiently estimate the original ratio matching objective. We perform experiments on density modeling over synthetic discrete data, graph generation, and training Ising models to evaluate our proposed method. The experimental results demonstrate that our method can significantly alleviate the limitations of ratio matching, perform more effectively in practice, and scale to high-dimensional problems. Our implementation is available at https://github.com/divelab/RMwGGIS.
