论文标题
功能区：使用多种云计算实例的成本效益和QoS感知的深度学习模型推断
RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
论文作者
论文摘要
深度学习模型推断是许多业务和科学发现过程中的关键服务。本文介绍了Ribbon，这是一种新颖的深度学习推理服务系统，符合两个相互竞争的目标：服务质量（QoS）目标和成本效益。功能区背后的关键思想是智能采用各种云计算实例（异质实例）来满足QoS目标并最大程度地节省成本。功能区设计了一种贝叶斯优化驱动的策略，该策略可帮助用户在云计算平台上为其模型推理服务需求构建最佳的异构实例集 - 而且，功能区展示了其优于使用均匀实例池的推理服务系统的优越性。功能区可为不同的学习模型节省多达16％的推理服务成本，包括新兴的深度学习建议系统模型和药物发现的启用模型。
Deep learning model inference is a key service in many businesses and scientific discovery processes. This paper introduces RIBBON, a novel deep learning inference serving system that meets two competing objectives: quality-of-service (QoS) target and cost-effectiveness. The key idea behind RIBBON is to intelligently employ a diverse set of cloud computing instances (heterogeneous instances) to meet the QoS target and maximize cost savings. RIBBON devises a Bayesian Optimization-driven strategy that helps users build the optimal set of heterogeneous instances for their model inference service needs on cloud computing platforms -- and, RIBBON demonstrates its superiority over existing approaches of inference serving systems using homogeneous instance pools. RIBBON saves up to 16% of the inference service cost for different learning models including emerging deep learning recommender system models and drug-discovery enabling models.
