论文标题
多代理强化学习中的奖励共享关系网络作为新兴行为的框架
Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior
论文作者
论文摘要
在这项工作中，我们通过用户定义的关系网络将“社交”相互作用集成到MARL设置中，并检查代理与代理关系对新兴行为兴起的影响。利用社会学和神经科学的见解，我们提出的框架模型使用了奖励共享关系网络（RSRN）的概念，其中网络边缘权重衡量了一个代理在成功（或“关心''）成功（或“关心”）的成功中。我们构建关系奖励是RSRN相互作用权重的函数，以通过多代理增强学习算法共同训练多代理系统。测试了具有不同关系网络结构（例如，自我利益，社区和专制网络）的3个代理方案的系统性能。我们的结果表明，奖励分享的关系网络可以显着影响学习的行为。我们认为，RSRN可以充当一个框架，不同的关系网络会产生独特的新兴行为，通常类似于对此类网络的直觉社会学理解。
In this work, we integrate `social' interactions into the MARL setup through a user-defined relational network and examine the effects of agent-agent relations on the rise of emergent behaviors. Leveraging insights from sociology and neuroscience, our proposed framework models agent relationships using the notion of Reward-Sharing Relational Networks (RSRN), where network edge weights act as a measure of how much one agent is invested in the success of (or `cares about') another. We construct relational rewards as a function of the RSRN interaction weights to collectively train the multi-agent system via a multi-agent reinforcement learning algorithm. The performance of the system is tested for a 3-agent scenario with different relational network structures (e.g., self-interested, communitarian, and authoritarian networks). Our results indicate that reward-sharing relational networks can significantly influence learned behaviors. We posit that RSRN can act as a framework where different relational networks produce distinct emergent behaviors, often analogous to the intuited sociological understanding of such networks.
