论文标题
非自主神经ODE的时间依赖性
Time Dependence in Non-Autonomous Neural ODEs
论文作者
论文摘要
神经普通微分方程（ODE）是深网的优雅重新解释，其中连续时间可以替代深度的离散概念，ode求解器执行正向传播，而伴随方法可以实现有效的，恒定的内存反向传播。神经ODES仅在非自主性时才是通用近似值，也就是说，动力学明确取决于时间。我们提出了一个新型的神经od家族，其中有时间依赖性是非参数，并且可以明确控制体重轨迹的平滑度，以允许表达和效率之间的折衷。利用这种增强的表现力，我们以速度和代表性能力均优于先前的神经ode变体，最终优于选定图像分类和视频预测任务上的标准重新NET和CNN模型。
Neural Ordinary Differential Equations (ODEs) are elegant reinterpretations of deep networks where continuous time can replace the discrete notion of depth, ODE solvers perform forward propagation, and the adjoint method enables efficient, constant memory backpropagation. Neural ODEs are universal approximators only when they are non-autonomous, that is, the dynamics depends explicitly on time. We propose a novel family of Neural ODEs with time-varying weights, where time-dependence is non-parametric, and the smoothness of weight trajectories can be explicitly controlled to allow a tradeoff between expressiveness and efficiency. Using this enhanced expressiveness, we outperform previous Neural ODE variants in both speed and representational capacity, ultimately outperforming standard ResNet and CNN models on select image classification and video prediction tasks.
