论文标题
为多机构增强学习提供的可解释行动
Explainable Action Advising for Multi-Agent Reinforcement Learning
论文作者
论文摘要
咨询行动是一种基于教师学生范式的强化学习的知识转移技术。专家老师在培训期间向学生提供建议，以提高学生的样本效率和政策表现。这种建议通常以国家行动对的形式给出。但是，这使学生很难推理并适用于新颖的状态。我们介绍了可解释的行动，建议教师提供行动建议以及相关的解释，以表明为什么选择了行动。这使学生能够自我反思自己学到的知识，实现建议的概括并提高样本效率和学习绩效，即使在老师最佳的环境中也是如此。我们从经验上表明，与最先进的方法相比，我们的框架在单一代理和多代理方案中都是有效的，从而提高了政策收益和收敛速度。
Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample efficiency and policy performance. Such advice is commonly given in the form of state-action pairs. However, it makes it difficult for the student to reason with and apply to novel states. We introduce Explainable Action Advising, in which the teacher provides action advice as well as associated explanations indicating why the action was chosen. This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal. We empirically show that our framework is effective in both single-agent and multi-agent scenarios, yielding improved policy returns and convergence rates when compared to state-of-the-art methods
