论文标题
多内核被动随机梯度算法和转移学习
Multi-kernel Passive Stochastic Gradient Algorithms and Transfer Learning
论文作者
论文摘要
本文开发了一种新型的被动随机梯度算法。在被动随机近似中，随机梯度算法无法控制评估成本函数嘈杂梯度的位置。经典的被动随机梯度算法使用近似Dirac Delta的内核来根据所需点的评估来称量梯度。在本文中，我们构建了多核无源随机梯度算法。该算法在高维问题中的性能要好得多，并结合了降低方差。我们分析了多内核算法的弱收敛及其收敛速率。在数值示例中，我们研究了用于传输学习的被动最小值平方（LMS）算法的多内核版本，以将性能与经典的被动版本进行比较。
This paper develops a novel passive stochastic gradient algorithm. In passive stochastic approximation, the stochastic gradient algorithm does not have control over the location where noisy gradients of the cost function are evaluated. Classical passive stochastic gradient algorithms use a kernel that approximates a Dirac delta to weigh the gradients based on how far they are evaluated from the desired point. In this paper we construct a multi-kernel passive stochastic gradient algorithm. The algorithm performs substantially better in high dimensional problems and incorporates variance reduction. We analyze the weak convergence of the multi-kernel algorithm and its rate of convergence. In numerical examples, we study the multi-kernel version of the passive least mean squares (LMS) algorithm for transfer learning to compare the performance with the classical passive version.
