论文标题
调整更聪明的不是更难：一种原则性的方法来调整浅网的学习率
Tune smarter not harder: A principled approach to tuning learning rates for shallow nets
论文作者
论文摘要
有效的高参数调整对于确保神经网络已闻名的性能至关重要。在这项工作中，提出了选择学习率的原则性方法，该方法是针对浅喂养神经网络的。我们将学习率与目标的梯度Lipschitz常数相关联，以最小化训练。提出了上述常数的上限，并且提出了一种始终导致非发散痕迹的搜索算法来利用派生的结合。通过模拟显示，所提出的搜索方法显着超过了现有的调谐方法，例如树parzen估计器（TPE）。提出的方法应用于三种不同的现有应用程序：a）OFDM系统中的渠道估计，b）汇率预测汇率和c）OFDM接收器中的汇率估算值，并且证明比使用相同或更少的计算功率的现有方法选择了更好的学习率。
Effective hyper-parameter tuning is essential to guarantee the performance that neural networks have come to be known for. In this work, a principled approach to choosing the learning rate is proposed for shallow feedforward neural networks. We associate the learning rate with the gradient Lipschitz constant of the objective to be minimized while training. An upper bound on the mentioned constant is derived and a search algorithm, which always results in non-divergent traces, is proposed to exploit the derived bound. It is shown through simulations that the proposed search method significantly outperforms the existing tuning methods such as Tree Parzen Estimators (TPE). The proposed method is applied to three different existing applications: a) channel estimation in OFDM systems, b) prediction of the exchange currency rates and c) offset estimation in OFDM receivers, and it is shown to pick better learning rates than the existing methods using the same or lesser compute power.
