论文标题
Wukong：无服务器并行计算的可扩展性增强框架
Wukong: A Scalable and Locality-Enhanced Framework for Serverless Parallel Computing
论文作者
论文摘要
无服务器计算越来越多地用于并行计算，这些计算传统上被用作状态应用程序。执行复杂的，爆发的，有向的无环图（DAG）作业对无服务器执行框架构成了一个主要挑战，该框架将需要快速扩展和安排高吞吐量的任务，同时最大程度地减少跨任务的数据移动。我们证明，对于无服务器并行计算，分散的计划使调度可以在可以并行安排任务的lambda执行者之间分发，并带来多个好处，包括增强的数据局部性，减少网络I/OS，自动资源弹性，自动资源弹性以及提高的成本效益。我们在AWS lambda上描述了我们的新无服务器并行框架的实现和部署，称为Wukong。我们表明，Wukong实现了近乎理想的可伸缩性，执行平行计算作业的速度最高68.17倍，将网络I/O减少了多个数量级，并且与NumpyWren相比，可节省92.96％的租户端成本。
Serverless computing is increasingly being used for parallel computing, which have traditionally been implemented as stateful applications. Executing complex, burst-parallel, directed acyclic graph (DAG) jobs poses a major challenge for serverless execution frameworks, which will need to rapidly scale and schedule tasks at high throughput, while minimizing data movement across tasks. We demonstrate that, for serverless parallel computations, decentralized scheduling enables scheduling to be distributed across Lambda executors that can schedule tasks in parallel, and brings multiple benefits, including enhanced data locality, reduced network I/Os, automatic resource elasticity, and improved cost effectiveness. We describe the implementation and deployment of our new serverless parallel framework, called Wukong, on AWS Lambda. We show that Wukong achieves near-ideal scalability, executes parallel computation jobs up to 68.17x faster, reduces network I/O by multiple orders of magnitude, and achieves 92.96% tenant-side cost savings compared to numpywren.
