论文标题
$^3 $ t：对准声学和文本进行培训和编辑预处理
A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing
论文作者
论文摘要
最近，语音表示学习改善了许多与语音有关的任务，例如语音识别，语音分类和语音到文本翻译。但是，以上所有任务都朝着语音理解的方向发展，但是对于反向方向，语音综合，由于产生高质量语音的挑战性质，代表性学习的潜力尚未实现。为了解决这个问题，我们提出了我们的框架，对准的声学文本预处理（$^3 $ t），该框架在训练过程中重建了带有文本输入和声学文本对齐的蒙面声信号。通过这种方式，预处理的模型可以生成高质量的重建光谱图，可以直接将其应用于语音编辑和看不见的扬声器tts。实验显示了$^3 $ t在语音编辑上的SOTA模型，并在没有外部说话者验证模型的情况下改善了多扬声器语音综合。
Recently, speech representation learning has improved many speech-related tasks such as speech recognition, speech classification, and speech-to-text translation. However, all the above tasks are in the direction of speech understanding, but for the inverse direction, speech synthesis, the potential of representation learning is yet to be realized, due to the challenging nature of generating high-quality speech. To address this problem, we propose our framework, Alignment-Aware Acoustic-Text Pretraining (A$^3$T), which reconstructs masked acoustic signals with text input and acoustic-text alignment during training. In this way, the pretrained model can generate high quality reconstructed spectrogram, which can be applied to the speech editing and unseen speaker TTS directly. Experiments show A$^3$T outperforms SOTA models on speech editing, and improves multi-speaker speech synthesis without the external speaker verification model.
