论文标题
使用TPU进行元学习的分布式演变策略
Distributed Evolution Strategies Using TPUs for Meta-Learning
论文作者
论文摘要
传统上，元学习通过整个任务依赖于反向传播，以迭代地改善模型的学习动态。但是，当缩放到复杂的任务时，这种方法在计算上是可悲的。我们建议使用张量处理单元（TPU）提出分布式进化元学习策略，该策略高度平行，可扩展到任意长的任务，而记忆成本没有增加。使用在Omniglot数据集上训练有进化策略的典型网络，我们在5次分类问题上获得了98.4％的精度。我们的算法使用的记忆使用比自动分化少40倍来计算梯度，因此，所得模型在反向传播训练的等效物（99.6％）的1.3％以内实现了准确性。我们观察到更好的分类准确性高达99.1％，较大的人口配置。我们进一步实验验证了在各种训练条件（不同的人口规模，模型大小，工人数量，射击，路，ES超参数等）中ES-Protonet的稳定性和性能。我们的贡献是双重的：我们在监督环境中对进化元学习进行了首次评估，并为TPU上的分布式进化策略创建了一般框架。
Meta-learning traditionally relies on backpropagation through entire tasks to iteratively improve a model's learning dynamics. However, this approach is computationally intractable when scaled to complex tasks. We propose a distributed evolutionary meta-learning strategy using Tensor Processing Units (TPUs) that is highly parallel and scalable to arbitrarily long tasks with no increase in memory cost. Using a Prototypical Network trained with evolution strategies on the Omniglot dataset, we achieved an accuracy of 98.4% on a 5-shot classification problem. Our algorithm used as much as 40 times less memory than automatic differentiation to compute the gradient, with the resulting model achieving accuracy within 1.3% of a backpropagation-trained equivalent (99.6%). We observed better classification accuracy as high as 99.1% with larger population configurations. We further experimentally validate the stability and performance of ES-ProtoNet across a variety of training conditions (varying population size, model size, number of workers, shot, way, ES hyperparameters, etc.). Our contributions are twofold: we provide the first assessment of evolutionary meta-learning in a supervised setting, and create a general framework for distributed evolution strategies on TPUs.
