论文标题
我们如何回答复杂的问题：长形答案的话语结构
How Do We Answer Complex Questions: Discourse Structure of Long-form Answers
论文作者
论文摘要
由多个句子组成的长格式答案可以为更广泛的问题提供细微和全面的答案。为了更好地理解这项复杂而研究的任务，我们研究了从三个数据集（ELI5，WebGPT和自然问题）收集的长形答案的功能结构。我们的主要目标是了解人类如何组织信息来制作复杂的答案。我们为长形答案开发了六个句子级功能角色的本体，并在640个答案段落中注释3.9k句子。不同的答案收集方法在不同的话语结构中表现出来。我们进一步分析了模型生成的答案 - 发现注释者在注释模型生成的答案与注释人写的答案相比彼此之间的同意较少。我们的注释数据使培训可以用于自动分析的强分类器。我们希望我们的工作能够激发对话语级建模和长期质量检查系统评估的未来研究。
Long-form answers, consisting of multiple sentences, can provide nuanced and comprehensive answers to a broader set of questions. To better understand this complex and understudied task, we study the functional structure of long-form answers collected from three datasets, ELI5, WebGPT and Natural Questions. Our main goal is to understand how humans organize information to craft complex answers. We develop an ontology of six sentence-level functional roles for long-form answers, and annotate 3.9k sentences in 640 answer paragraphs. Different answer collection methods manifest in different discourse structures. We further analyze model-generated answers -- finding that annotators agree less with each other when annotating model-generated answers compared to annotating human-written answers. Our annotated data enables training a strong classifier that can be used for automatic analysis. We hope our work can inspire future research on discourse-level modeling and evaluation of long-form QA systems.
