论文标题
AdaCat：自回归模型的自适应分类离散化
AdaCat: Adaptive Categorical Discretization for Autoregressive Models
论文作者
论文摘要
自回归生成模型可以估计复杂的连续数据分布，例如在RL环境，图像强度和音频中的轨迹推出。大多数最先进的模型将连续数据离散为几个箱，并在箱上使用分类分布来近似连续数据分布。优势在于，分类分布可以轻松地表达多种模式，并且可以简单地进行优化。但是，如果不使用更多的垃圾箱，这种近似就无法表达密度的急剧变化，从而使其参数效率低下。我们提出了一种称为自适应分类离散化（ADACAT）的有效，表达，多模式的参数化。 AdaCat自适应地自适应地自动回收模型的每个维度，这使该模型能够分配密度为良好的感兴趣间隔，从而提高了参数效率。 Adacat概括了分类和基于分数的回归。 Adacat是任何基于离散化的分布估计器的简单附加组件。在实验中，Adacat改善了现实世界表数据，图像，音频和轨迹的密度估计，并改善了基于模型的离线RL计划。
Autoregressive generative models can estimate complex continuous data distributions, like trajectory rollouts in an RL environment, image intensities, and audio. Most state-of-the-art models discretize continuous data into several bins and use categorical distributions over the bins to approximate the continuous data distribution. The advantage is that the categorical distribution can easily express multiple modes and are straightforward to optimize. However, such approximation cannot express sharp changes in density without using significantly more bins, making it parameter inefficient. We propose an efficient, expressive, multimodal parameterization called Adaptive Categorical Discretization (AdaCat). AdaCat discretizes each dimension of an autoregressive model adaptively, which allows the model to allocate density to fine intervals of interest, improving parameter efficiency. AdaCat generalizes both categoricals and quantile-based regression. AdaCat is a simple add-on to any discretization-based distribution estimator. In experiments, AdaCat improves density estimation for real-world tabular data, images, audio, and trajectories, and improves planning in model-based offline RL.
