论文标题
可区分的编程àlaMoreau
Differentiable Programming à la Moreau
论文作者
论文摘要
Moreau Invelope的概念对于分析机器学习的一阶优化算法至关重要。但是，尚未开发和扩展，以应用于深层网络，更广泛地应用于具有可区分编程实现的机器学习系统。我们定义了一个适合Moreau信封的构图积分，并显示如何将其集成到可区分的编程中。所提出的框架在数学优化框架中铸造了与虚拟目标传播的概念有关的梯度背部传播的几种变体。
The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning. Yet, it has not been developed and extended to be applied to a deep network and, more broadly, to a machine learning system with a differentiable programming implementation. We define a compositional calculus adapted to Moreau envelopes and show how to integrate it within differentiable programming. The proposed framework casts in a mathematical optimization framework several variants of gradient back-propagation related to the idea of the propagation of virtual targets.
