论文标题
法律提示：教语言模型像律师一样思考
Legal Prompting: Teaching a Language Model to Think Like a Lawyer
论文作者
论文摘要
能够零或几次促使方法的大型语言模型引起了迅速工程的新研究领域。最近的进步表明，例如，思想链（COT）提示可以显着改善算术或常识任务。我们探讨了这种方法如何处理法律推理任务，并根据日本律师律师考试来测试零射击/少量射击和微调方法。我们的发现表明，尽管COT提示和通过解释方法进行微调显示了改进，但最佳结果是由提示产生的，这些提示来自特定的法律推理技术，例如IRAC（问题，规则，应用程序，结论）。根据我们的实验，我们将2021最佳结果从0.7037的精度提高到0.8148的精度，并以0.7431的精度击败2022最佳系统。
Large language models that are capable of zero or few-shot prompting approaches have given rise to the new research area of prompt engineering. Recent advances showed that for example Chain-of-Thought (CoT) prompts can improve arithmetic or common sense tasks significantly. We explore how such approaches fare with legal reasoning tasks and take the COLIEE entailment task based on the Japanese Bar exam for testing zero-shot/few-shot and fine-tuning approaches. Our findings show that while CoT prompting and fine-tuning with explanations approaches show improvements, the best results are produced by prompts that are derived from specific legal reasoning techniques such as IRAC (Issue, Rule, Application, Conclusion). Based on our experiments we improve the 2021 best result from 0.7037 accuracy to 0.8148 accuracy and beat the 2022 best system of 0.6789 accuracy with an accuracy of 0.7431.
