论文标题
与Hessian近似的控制标准的随机阻尼L-BFGS
Stochastic Damped L-BFGS with Controlled Norm of the Hessian Approximation
论文作者
论文摘要
我们提出了一种新的随机方差降低阻尼L-BFGS算法，在其中，我们利用了Hessian近似值最大和最小的特征值的边界估算，以平衡其质量和调理。我们的算法Varchen借鉴了以前的工作，该作品提出了一种称为SDLBFGS的新型随机阻尼L-BFGS算法。我们几乎确定收敛到固定点和复杂性结合。我们从经验上证明，在修改后的DavidNet问题上，Varchen比SDLBFGS-VR和SVRG更强大 - 在深度学习的背景下，出现的一个高度非概念和条件不足的问题，它们的性能与逻辑回归问题和非concevex支持者的机器机器问题相当。
We propose a new stochastic variance-reduced damped L-BFGS algorithm, where we leverage estimates of bounds on the largest and smallest eigenvalues of the Hessian approximation to balance its quality and conditioning. Our algorithm, VARCHEN, draws from previous work that proposed a novel stochastic damped L-BFGS algorithm called SdLBFGS. We establish almost sure convergence to a stationary point and a complexity bound. We empirically demonstrate that VARCHEN is more robust than SdLBFGS-VR and SVRG on a modified DavidNet problem -- a highly nonconvex and ill-conditioned problem that arises in the context of deep learning, and their performance is comparable on a logistic regression problem and a nonconvex support-vector machine problem.
