论文标题
通过管道并行分析图神经网络的性能
Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
论文作者
论文摘要
可以通过图形描述许多有趣的数据集和机器学习和深度学习的数据集。随着图形结构数据集的规模和复杂性的增加，例如在扩展的社交网络，蛋白质折叠，化学相互作用网络和材料相变，提高了应用于这些的机器学习技术的效率至关重要。在这项研究中，我们专注于图形神经网络（GNN），这些神经网络（GNN）在诸如节点或边缘分类和链接预测等任务中取得了巨大成功。但是，由于通过密集的图形关系执行的必要递归计算，标准GNN模型具有缩放限制，从而导致内存和运行时瓶颈。虽然需要新的处理较大网络来推进图形技术，并且已经提出了一些方法，但我们研究了如何使用现有的工具和框架在深度学习社区中取得成功。特别是，我们调查了Google在2018年推出的GPIPE将管道并行应用于GNN模型。
Many interesting datasets ubiquitous in machine learning and deep learning can be described via graphs. As the scale and complexity of graph-structured datasets increase, such as in expansive social networks, protein folding, chemical interaction networks, and material phase transitions, improving the efficiency of the machine learning techniques applied to these is crucial. In this study, we focus on Graph Neural Networks (GNN) that have found great success in tasks such as node or edge classification and link prediction. However, standard GNN models have scaling limits due to necessary recursive calculations performed through dense graph relationships that lead to memory and runtime bottlenecks. While new approaches for processing larger networks are needed to advance graph techniques, and several have been proposed, we study how GNNs could be parallelized using existing tools and frameworks that are known to be successful in the deep learning community. In particular, we investigate applying pipeline parallelism to GNN models with GPipe, introduced by Google in 2018.
