论文标题
fugashi，一种用于python的日语的工具
fugashi, a Tool for Tokenizing Japanese in Python
论文作者
论文摘要
近年来，大规模多语言NLP项目的数量有所增加。但是，即使在此类项目中，通常会排除具有特殊处理要求的语言。一种这样的语言就是日语。日语是没有空间的书写，象征化是非平凡的，尽管存在高质量的开源标记，但它们可能很难使用，并且缺乏英语文档。本文介绍了Python的Mecab包装器Fugashi，并介绍了日语的象征。
Recent years have seen an increase in the number of large-scale multilingual NLP projects. However, even in such projects, languages with special processing requirements are often excluded. One such language is Japanese. Japanese is written without spaces, tokenization is non-trivial, and while high quality open source tokenizers exist they can be hard to use and lack English documentation. This paper introduces fugashi, a MeCab wrapper for Python, and gives an introduction to tokenizing Japanese.
