论文标题

带有Sig-Transformer编码器的瑞典医疗处方中的信息提取

Information Extraction from Swedish Medical Prescriptions with Sig-Transformer Encoder

论文作者

Biyong, John Pougue, Wang, Bo, Lyons, Terry, Nevado-Holgado, Alejo J

论文摘要

依靠大型审慎的语言模型(例如来自变形金刚(BERT)的双向编码器表示)来编码和添加简单的预测层,这导致了许多临床自然语言处理(NLP)任务的令人印象深刻的性能。在这项工作中,我们通过将签名变换与自我发病模型结合在一起,向变压器体系结构展示了新颖的扩展。该体系结构是在嵌入和预测层之间添加的。新瑞典处方数据上的实验表明,与基线模型相比,在三个信息提取任务中的两个中,提出的架构将在三个信息中提出。最后,我们评估了应用多语言BERT和将瑞典语翻译成英语的两种不同的嵌入方法,然后用临床注释预计的BERT模型进行编码。

Relying on large pretrained language models such as Bidirectional Encoder Representations from Transformers (BERT) for encoding and adding a simple prediction layer has led to impressive performance in many clinical natural language processing (NLP) tasks. In this work, we present a novel extension to the Transformer architecture, by incorporating signature transform with the self-attention model. This architecture is added between embedding and prediction layers. Experiments on a new Swedish prescription data show the proposed architecture to be superior in two of the three information extraction tasks, comparing to baseline models. Finally, we evaluate two different embedding approaches between applying Multilingual BERT and translating the Swedish text to English then encode with a BERT model pretrained on clinical notes.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源