论文标题
致力于值得信赖的神经程序综合
Toward Trustworthy Neural Program Synthesis
论文作者
论文摘要
我们开发了一种方法来估计从大语言模型中采样程序的可能性是正确的。鉴于对编程问题的自然语言描述，我们的方法均样本既定候选程序，又要候选人指定该程序应如何行为。这允许学习一个模型，该模型构成了程序正确性的概率预测。我们的系统还注入了谓词对解释生成代码的行为有用的，而人类在人类研究中偏爱这些代码而不是原始语言模型的输出。我们的方法简单，易于实施，并保持最新生成精度的结果。
We develop an approach to estimate the probability that a program sampled from a large language model is correct. Given a natural language description of a programming problem, our method samples both candidate programs as well as candidate predicates specifying how the program should behave. This allows learning a model that forms a well-calibrated probabilistic prediction of program correctness. Our system also infers which predicates are useful to explain the behavior of the generated code, and humans preferred these in a human study over raw language model outputs. Our method is simple, easy to implement, and maintains state of the art generation accuracy results.
