论文标题
关于文本生成中质量多样性评估与分配拟合目标之间的关系
On the Relation between Quality-Diversity Evaluation and Distribution-Fitting Goal in Text Generation
论文作者
论文摘要
文本生成模型的目的是符合文本的基本实际概率分布。为了评估绩效，通常应用质量和多样性指标。但是，尚不清楚质量多样性评估可以反映出分配拟合的目标。在本文中，我们试图以一种理论方法来揭示这种关系。我们证明，在某些条件下，质量和多样性的线性组合构成了生成的分布与实际分布之间的分歧度量。我们还表明，常用的bleu/selfbleu度量对无法匹配任何分歧度量，因此提出CR/NRR作为质量/多样性度量对的替代品。
The goal of text generation models is to fit the underlying real probability distribution of text. For performance evaluation, quality and diversity metrics are usually applied. However, it is still not clear to what extend can the quality-diversity evaluation reflect the distribution-fitting goal. In this paper, we try to reveal such relation in a theoretical approach. We prove that under certain conditions, a linear combination of quality and diversity constitutes a divergence metric between the generated distribution and the real distribution. We also show that the commonly used BLEU/Self-BLEU metric pair fails to match any divergence metric, thus propose CR/NRR as a substitute for quality/diversity metric pair.
