论文标题
使用句子级语言模型迈向更好的故事情节
Toward Better Storylines with Sentence-Level Language Models
论文作者
论文摘要
我们提出了一个句子级的语言模型，该模型从有限的流利替代方案中选择了一个故事中的下一个句子。由于它不需要对流利度进行建模，因此句子级语言模型可以集中在较长的范围依赖性上，这对于多句子连贯至关重要。我们的方法并没有处理单个单词，而是将故事视为预先训练的句子嵌入列表，并预测下一个句子的嵌入，这比预测单词嵌入更有效。值得注意的是，这使我们可以在培训期间考虑大量候选人。我们以最先进的精度在无监督的故事任务上证明了我们的方法的有效性，并在下一个句子预测任务上表现出了令人鼓舞的结果。
We propose a sentence-level language model which selects the next sentence in a story from a finite set of fluent alternatives. Since it does not need to model fluency, the sentence-level language model can focus on longer range dependencies, which are crucial for multi-sentence coherence. Rather than dealing with individual words, our method treats the story so far as a list of pre-trained sentence embeddings and predicts an embedding for the next sentence, which is more efficient than predicting word embeddings. Notably this allows us to consider a large number of candidates for the next sentence during training. We demonstrate the effectiveness of our approach with state-of-the-art accuracy on the unsupervised Story Cloze task and with promising results on larger-scale next sentence prediction tasks.
