论文标题
了解自我监督音频变压器的自我注意
Understanding Self-Attention of Self-Supervised Audio Transformers
论文作者
论文摘要
自我监督的音频变压器（SAT）在许多下游语音应用程序等许多下游语音应用中取得了巨大的成功，但是尚未广泛探索它们的工作方式。在这项工作中，我们提出了多种策略来分析SAT中的注意机制。我们将注意力分类为可解释的类别，在该类别中，我们发现每个类别都具有其独特的功能。我们提供了一个可视化工具，用于了解多头自我注意力，重要性排名识别批判性关注的重要性排名策略以及提高模型绩效的注意力完善技术。
Self-supervised Audio Transformers (SAT) enable great success in many downstream speech applications like ASR, but how they work has not been widely explored yet. In this work, we present multiple strategies for the analysis of attention mechanisms in SAT. We categorize attentions into explainable categories, where we discover each category possesses its own unique functionality. We provide a visualization tool for understanding multi-head self-attention, importance ranking strategies for identifying critical attention, and attention refinement techniques to improve model performance.
