论文标题
共形非政策预测
Conformal Off-policy Prediction
论文作者
论文摘要
在许多应用程序中，在线部署之前需要离线评估新政策，因此非政策评估至关重要。大多数现有方法都集中在预期的回报上，通过平均定义目标参数，仅提供点估计器。在本文中，我们开发了一种新的程序，以从任何初始状态开始为目标策略的回报产生可靠的间隔估计器。我们的提案说明了回报围绕其期望的可变性，重点关注个人效应，并提供有效的不确定性量化。我们的主要思想在于设计伪策略，该伪政策会产生子样本，就好像是从目标策略中取样一样，以便现有的保形预测算法适用于预测间隔构建。我们的方法是由来自短视频平台的理论，合成数据和真实数据证明是合理的。
Off-policy evaluation is critical in a number of applications where new policies need to be evaluated offline before online deployment. Most existing methods focus on the expected return, define the target parameter through averaging and provide a point estimator only. In this paper, we develop a novel procedure to produce reliable interval estimators for a target policy's return starting from any initial state. Our proposal accounts for the variability of the return around its expectation, focuses on the individual effect and offers valid uncertainty quantification. Our main idea lies in designing a pseudo policy that generates subsamples as if they were sampled from the target policy so that existing conformal prediction algorithms are applicable to prediction interval construction. Our methods are justified by theories, synthetic data and real data from short-video platforms.
