论文标题
符号语言的绑定语言模型
Binding Language Models in Symbolic Languages
论文作者
论文摘要
尽管最近端到端的神经方法在性能和易用性方面都在主导NLP任务，但它们缺乏可解释性和鲁棒性。 We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e.g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and （3）仅需要少数文字示例注释。具体来说，我们将GPT-3法典作为LM。在解析阶段，只有少数文本的示例，Codex能够识别任务输入的一部分，而原始编程语言无法对此表示无法掌握，并且可以正确生成API调用以提示Codex求解无法回述的部分，并确定在与原始语法兼容的同时识别API呼叫的位置。在执行阶段，法典可以在API调用中执行多功能功能（例如，常识质量质量质量提取，信息提取）。 Binder通过有利于人类调试的明确输出程序来实现WikableQuestions和TabFact数据集的最新结果。请注意，以前的最佳系统都在数以万计的特定于任务的样本上进行了填充，而活页夹仅将数十个注释用作文本的范例，而无需任何培训。我们的代码可在https://github.com/hkunlp/binder上找到。
Though end-to-end neural approaches have recently been dominating NLP tasks in both performance and ease-of-use, they lack interpretability and robustness. We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e.g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations. Specifically, we employ GPT-3 Codex as the LM. In the parsing stage, with only a few in-context exemplars, Codex is able to identify the part of the task input that cannot be answerable by the original programming language, correctly generate API calls to prompt Codex to solve the unanswerable part, and identify where to place the API calls while being compatible with the original grammar. In the execution stage, Codex can perform versatile functionalities (e.g., commonsense QA, information extraction) given proper prompts in the API calls. Binder achieves state-of-the-art results on WikiTableQuestions and TabFact datasets, with explicit output programs that benefit human debugging. Note that previous best systems are all finetuned on tens of thousands of task-specific samples, while Binder only uses dozens of annotations as in-context exemplars without any training. Our code is available at https://github.com/HKUNLP/Binder .
