论文标题
上下文BERT：使用全局状态调节语言模型
Contextual BERT: Conditioning the Language Model Using a Global State
论文作者
论文摘要
伯特（Bert）是一种流行的语言模型，其主要的预训练任务是填写空白，即根据其余单词预测一个被句子掩盖的单词。但是，在某些应用程序中，拥有额外的上下文可以帮助模型通过考虑域或写作时间来做出正确的预测。这激发了我们通过在固定尺寸上下文上添加用于调理的全球状态来推动BERT体系结构。我们介绍了两种新颖的方法，并将其应用于行业用例，在那里我们将缺少文章的时尚服装以特定的客户为条件。与文献中其他方法的实验比较表明，我们的方法显着改善了个性化。
BERT is a popular language model whose main pre-training task is to fill in the blank, i.e., predicting a word that was masked out of a sentence, based on the remaining words. In some applications, however, having an additional context can help the model make the right prediction, e.g., by taking the domain or the time of writing into account. This motivates us to advance the BERT architecture by adding a global state for conditioning on a fixed-sized context. We present our two novel approaches and apply them to an industry use-case, where we complete fashion outfits with missing articles, conditioned on a specific customer. An experimental comparison to other methods from the literature shows that our methods improve personalization significantly.
