论文标题
因果关系，以改善增强学习的鲁棒性
Causal Counterfactuals for Improving the Robustness of Reinforcement Learning
论文作者
论文摘要
增强学习（RL）用于各种机器人应用中。 RL使代理可以通过与环境互动来自主学习任务。任务越重要，对RL系统鲁棒性的需求越高。因果RL结合了RL和因果推断，使RL更健壮。因果RL药物使用因果表示来捕获可以从一个任务转移到另一个任务的不变因果机制。当前，在因果RL方面的研究有限，现有解决方案通常不完整或不可行。在这项工作中，我们提出了CausalCF，这是第一个完整的因果RL解决方案，结合了因果关系和Cophy的想法。因果好奇心提供了一种使用干预措施的方法，并修改了Cophy以使RL药物能够执行反事实。因果关系已应用于因果关系中的机器人抓握和操纵任务。 Causalworld提供了基于Trifinger机器人的逼真的模拟环境。我们将CausalCF应用于复杂的机器人任务，并表明它使用Causalworld提高了RL代理的鲁棒性。
Reinforcement learning (RL) is used in various robotic applications. RL enables agents to learn tasks autonomously by interacting with the environment. The more critical the tasks are, the higher the demand for the robustness of the RL systems. Causal RL combines RL and causal inference to make RL more robust. Causal RL agents use a causal representation to capture the invariant causal mechanisms that can be transferred from one task to another. Currently, there is limited research in Causal RL, and existing solutions are usually not complete or feasible for real-world applications. In this work, we propose CausalCF, the first complete Causal RL solution incorporating ideas from Causal Curiosity and CoPhy. Causal Curiosity provides an approach for using interventions, and CoPhy is modified to enable the RL agent to perform counterfactuals. Causal Curiosity has been applied to robotic grasping and manipulation tasks in CausalWorld. CausalWorld provides a realistic simulation environment based on the TriFinger robot. We apply CausalCF to complex robotic tasks and show that it improves the RL agent's robustness using CausalWorld.
