论文标题
深度学习优化算法的比较
A Comparison of Optimization Algorithms for Deep Learning
论文作者
论文摘要
近年来，我们目睹了深度学习的兴起。深层神经网络已经证明了它们在许多领域的成功。但是，随着神经网络的进度和数据集变得更大，这些网络的优化变得越来越困难。因此，在过去几年中提出了更高级的优化算法。在这项研究中，详细研究了广泛使用的深度学习优化算法。为此，这些称为自适应梯度方法的算法都是针对受监督和无监督任务实现的。通过指出它们针对基本优化算法的差异，比较了训练过程中算法和在四个图像数据集中的算法和结果的结果。
In recent years, we have witnessed the rise of deep learning. Deep neural networks have proved their success in many areas. However, the optimization of these networks has become more difficult as neural networks going deeper and datasets becoming bigger. Therefore, more advanced optimization algorithms have been proposed over the past years. In this study, widely used optimization algorithms for deep learning are examined in detail. To this end, these algorithms called adaptive gradient methods are implemented for both supervised and unsupervised tasks. The behaviour of the algorithms during training and results on four image datasets, namely, MNIST, CIFAR-10, Kaggle Flowers and Labeled Faces in the Wild are compared by pointing out their differences against basic optimization algorithms.
