论文标题
无偏梯度估计分配稳健学习
Unbiased Gradient Estimation for Distributionally Robust Learning
论文作者
论文摘要
为了改善模型概括，我们考虑了一种基于分布鲁棒学习（DRL）的新方法，该方法将随机梯度下降应用于外部最小化问题。我们的算法通过多级蒙特卡洛随机化有效地估算了内部最大化问题的梯度。利用理论结果阐明了标准梯度估计器为何失败，我们建立了方法的梯度估计器的最佳参数化，以平衡计算时间和统计差异之间的基本权衡。数值实验表明，我们的DRL方法比以前的工作产生了重大好处。
Seeking to improve model generalization, we consider a new approach based on distributionally robust learning (DRL) that applies stochastic gradient descent to the outer minimization problem. Our algorithm efficiently estimates the gradient of the inner maximization problem through multi-level Monte Carlo randomization. Leveraging theoretical results that shed light on why standard gradient estimators fail, we establish the optimal parameterization of the gradient estimators of our approach that balances a fundamental tradeoff between computation time and statistical variance. Numerical experiments demonstrate that our DRL approach yields significant benefits over previous work.
