论文标题
基于群的非凸优化的梯度下降方法
Swarm-Based Gradient Descent Method for Non-Convex Optimization
论文作者
论文摘要
我们引入了一种新的基于群的梯度下降（SBGD）方法，以进行非凸优化。群由代理组成，每个代理都有一个位置，$ {\ mathbf x} $和质量，$ m $。他们动态的关键是交流：质量正在从高地面的代理转移到低（最）。同时，代理商以步长，$ h = h（{\ mathbf x}，m）$更改位置，调整了其相对质量：较重的代理在本地梯度方向上进行较小的时间步长，而较轻的代理商则根据回溯协议进行更大的时间步骤。因此，人群在“较重”的领导者之间动态分配，预计将接近当地的最小值和“较轻”的探险家。借助其大型协议，探险家有望在群体中遇到改善的位置。如果他们这样做，那么他们扮演“沉重”群领导人的角色等等。一，二维和20维基准中的收敛分析和数值模拟证明了SBGD作为全局优化器的有效性。
We introduce a new Swarm-Based Gradient Descent (SBGD) method for non-convex optimization. The swarm consists of agents, each is identified with a position, ${\mathbf x}$, and mass, $m$. The key to their dynamics is communication: masses are being transferred from agents at high ground to low(-est) ground. At the same time, agents change positions with step size, $h=h({\mathbf x},m)$, adjusted to their relative mass: heavier agents proceed with small time-steps in the direction of local gradient, while lighter agents take larger time-steps based on a backtracking protocol. Accordingly, the crowd of agents is dynamically divided between `heavier' leaders, expected to approach local minima, and `lighter' explorers. With their large-step protocol, explorers are expected to encounter improved position for the swarm; if they do, then they assume the role of `heavy' swarm leaders and so on. Convergence analysis and numerical simulations in one-, two-, and 20-dimensional benchmarks demonstrate the effectiveness of SBGD as a global optimizer.
