论文标题
从文本环境的场景图中的常识知识
Commonsense Knowledge from Scene Graphs for Textual Environments
论文作者
论文摘要
基于文本的游戏正常用于增强学习作为现实世界模拟环境。它们通常是不完美的信息游戏，它们的互动仅在文本模式中。为了挑战这些游戏，通过在游戏外提供知识（例如人类常识）来补充缺失的信息。但是，此类知识仅从以前的作品中的文本信息中获得。在本文中，我们研究了采用从视觉数据集（例如场景图数据集）获得的常识性推理的优势。通常，与人类的文本相比，图像传达了更全面的信息。该属性使提取常识性关系知识可用于在游戏中有效地表现更有用。我们比较了视觉基因组（场景图数据集）和ConceptNet（基于文本的知识）中可用的空间关系的统计数据，以分析引入场景图数据集的有效性。我们还对需要常识性推理的基于文本的游戏任务进行了实验。我们的实验结果表明，与现有的最新方法相比，我们提出的方法具有更高和竞争性的性能。
Text-based games are becoming commonly used in reinforcement learning as real-world simulation environments. They are usually imperfect information games, and their interactions are only in the textual modality. To challenge these games, it is effective to complement the missing information by providing knowledge outside the game, such as human common sense. However, such knowledge has only been available from textual information in previous works. In this paper, we investigate the advantage of employing commonsense reasoning obtained from visual datasets such as scene graph datasets. In general, images convey more comprehensive information compared with text for humans. This property enables to extract commonsense relationship knowledge more useful for acting effectively in a game. We compare the statistics of spatial relationships available in Visual Genome (a scene graph dataset) and ConceptNet (a text-based knowledge) to analyze the effectiveness of introducing scene graph datasets. We also conducted experiments on a text-based game task that requires commonsense reasoning. Our experimental results demonstrated that our proposed methods have higher and competitive performance than existing state-of-the-art methods.
