论文标题
了解一类分散和联合的优化算法：多率反馈控制观点
Understanding A Class of Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspective
论文作者
论文摘要
分布式算法在许多应用程序中都起着越来越重要的作用，例如机器学习，信号处理和控制。大量的研究工作致力于开发和分析各种应用的新算法。在这项工作中，我们提供了一个新的观点，可以理解，分析和设计分布式优化算法。通过多速率反馈控制的镜头，我们表明，包括流行的分散/联合方案在内的广泛的分布式算法可以看作是分散的一定连续的时间反馈控制系统，可能具有多个采样率，例如分散的梯度下降，梯度跟踪，梯度跟踪，梯度跟踪，渐变，逐渐融合。这个关键观察不仅允许我们开发一个通用框架来分析整个算法类的收敛性。更重要的是，这也导致了设计新的分布式算法的有趣方法。我们发展了框架背后的理论，并提供了示例，以突出如何在实践中使用该框架。
Distributed algorithms have been playing an increasingly important role in many applications such as machine learning, signal processing, and control. Significant research efforts have been devoted to developing and analyzing new algorithms for various applications. In this work, we provide a fresh perspective to understand, analyze, and design distributed optimization algorithms. Through the lens of multi-rate feedback control, we show that a wide class of distributed algorithms, including popular decentralized/federated schemes, can be viewed as discretizing a certain continuous-time feedback control system, possibly with multiple sampling rates, such as decentralized gradient descent, gradient tracking, and federated averaging. This key observation not only allows us to develop a generic framework to analyze the convergence of the entire algorithm class. More importantly, it also leads to an interesting way of designing new distributed algorithms. We develop the theory behind our framework and provide examples to highlight how the framework can be used in practice.
