论文标题
总结虚拟机器人过去的自然语言行动
Summarizing a virtual robot's past actions in natural language
论文作者
论文摘要
我们提出并展示了在虚拟环境中提供自然语言摘要的自然语言摘要的任务。我们解释了为什么这样的任务很重要，是什么使它变得困难，并讨论了如何解决。为了鼓励其他人对此进行努力，我们展示了如何将适用于指导后完成任务的自然语言描述与机器人动作相匹配的流行现有数据集，以作为机器人动作摘要工作的培训理由。我们提出并测试了几种学习方法来生成此类摘要的方法，从自动计划者使用的动作的机器人的以自动视频框架或中间文本表示开始。我们对结果进行定量和定性评估，这可以作为未来工作的基准。
We propose and demonstrate the task of giving natural language summaries of the actions of a robotic agent in a virtual environment. We explain why such a task is important, what makes it difficult, and discuss how it might be addressed. To encourage others to work on this, we show how a popular existing dataset that matches robot actions with natural language descriptions designed for an instruction following task can be repurposed to serve as a training ground for robot action summarization work. We propose and test several methods of learning to generate such summaries, starting from either egocentric video frames of the robot taking actions or intermediate text representations of the actions used by an automatic planner. We provide quantitative and qualitative evaluations of our results, which can serve as a baseline for future work.
