论文标题
近端降低随机梯度方法的采样和更新频率
Sampling and Update Frequencies in Proximal Variance-Reduced Stochastic Gradient Methods
论文作者
论文摘要
近来，降低方差的随机梯度方法已越来越受欢迎。有几种变体具有用于梯度存储和采样的不同策略，这项工作涉及这两个方面之间的相互作用。我们提出了一种一般的近端降低梯度方法，并在较强的凸度假设下进行分析。该算法的特殊情况包括传奇，L-SVRG及其近端变体。我们的分析阐明了时代长度的选择，并需要平衡迭代的收敛与梯度的频率存储。分析改善了文献中发现的其他收敛速率，并为传奇产生了一种新的，更快的收敛采样策略。预测利率与实际利率相同的问题实例与基于现实世界数据的问题一起提出。
Variance-reduced stochastic gradient methods have gained popularity in recent times. Several variants exist with different strategies for the storing and sampling of gradients and this work concerns the interactions between these two aspects. We present a general proximal variance-reduced gradient method and analyze it under strong convexity assumptions. Special cases of the algorithm include SAGA, L-SVRG and their proximal variants. Our analysis sheds light on epoch-length selection and the need to balance the convergence of the iterates with how often gradients are stored. The analysis improves on other convergence rates found in the literature and produces a new and faster converging sampling strategy for SAGA. Problem instances for which the predicted rates are the same as the practical rates are presented together with problems based on real world data.
