论文标题
Redwood：使用碰撞检测来生长大规模的意图分类数据集
Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset
论文作者
论文摘要
对话系统必须能够随着时间的推移通过更新来纳入新技能，以反映新的用例或部署方案。同样，此类ML驱动系统的开发人员需要能够在已经存在的数据集中添加新的培训数据，以支持这些新技能。在意图分类系统中，如果对新技能的意图训练数据的语义与已经存在的意图重叠，则可能会出现问题。我们称此类案件发生冲突。本文介绍了多个数据集之间意图碰撞检测的任务，以提高系统的技能。我们介绍了几种检测碰撞的方法，并评估我们在显示碰撞的真实数据集上的方法。为了强调对意图碰撞检测的需求，我们表明，如果添加新数据，则模型性能会受到影响。最后，我们使用碰撞检测来构建和基准一个新的数据集Redwood，该数据集由13个原始意图分类数据集的451个Nentent类别组成，使其成为最大的公开可用意图分类基准。
Dialog systems must be capable of incorporating new skills via updates over time in order to reflect new use cases or deployment scenarios. Similarly, developers of such ML-driven systems need to be able to add new training data to an already-existing dataset to support these new skills. In intent classification systems, problems can arise if training data for a new skill's intent overlaps semantically with an already-existing intent. We call such cases collisions. This paper introduces the task of intent collision detection between multiple datasets for the purposes of growing a system's skillset. We introduce several methods for detecting collisions, and evaluate our methods on real datasets that exhibit collisions. To highlight the need for intent collision detection, we show that model performance suffers if new data is added in such a way that does not arbitrate colliding intents. Finally, we use collision detection to construct and benchmark a new dataset, Redwood, which is composed of 451 ntent categories from 13 original intent classification datasets, making it the largest publicly available intent classification benchmark.
