论文标题
线性注意耦合傅里叶神经操作员用于模拟三维湍流
Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence
论文作者
论文摘要
通过神经网络对三维（3D）的湍流进行建模很困难，因为3D湍流是高度非线性的，具有高度的自由度，相应的模拟是内存密集的。最近，注意机制已被证明是提高神经网络在湍流模拟中的性能的有前途的方法。但是，对于输入尺寸$ n $，标准的自我发项机制使用$ o（n^2）$时间和空间，而这种二次复杂性已成为主要瓶颈，以便将注意力应用于3D湍流模拟。在这项工作中，我们通过线性注意网络的概念解决了这个问题。线性注意通过添加两个线性预测来近似标准的注意力，从而将整体自我发项的复杂性从$ O（n^2）$降低到时间和空间中的$ O（n）$。线性注意耦合傅里叶神经操作员（LAFNO）是为3D湍流的模拟而开发的。数值模拟表明，线性注意机制以相同水平的计算成本提供40 \％的误差降低，Lafno可以准确地重建各种统计数据和3D湍流的瞬时空间结构。线性注意力方法将有助于改善3D非线性问题的神经网络模型，该模型涉及其他科学领域中的高维数据。
Modeling three-dimensional (3D) turbulence by neural networks is difficult because 3D turbulence is highly-nonlinear with high degrees of freedom and the corresponding simulation is memory-intensive. Recently, the attention mechanism has been shown as a promising approach to boost the performance of neural networks on turbulence simulation. However, the standard self-attention mechanism uses $O(n^2)$ time and space with respect to input dimension $n$, and such quadratic complexity has become the main bottleneck for attention to be applied on 3D turbulence simulation. In this work, we resolve this issue with the concept of linear attention network. The linear attention approximates the standard attention by adding two linear projections, reducing the overall self-attention complexity from $O(n^2)$ to $O(n)$ in both time and space. The linear attention coupled Fourier neural operator (LAFNO) is developed for the simulation of 3D turbulence. Numerical simulations show that the linear attention mechanism provides 40\% error reduction at the same level of computational cost, and LAFNO can accurately reconstruct a variety of statistics and instantaneous spatial structures of 3D turbulence. The linear attention method would be helpful for the improvement of neural network models of 3D nonlinear problems involving high-dimensional data in other scientific domains.
