ANODEV2: A Coupled Neural ODE Framework
ANODEV2: A Coupled Neural ODE Framework
复制标题
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Tianjun Zhang;Z. Yao;A. Gholami;Joseph Gonzalez;K. Keutzer;Michael W. Mahoney;G. Biros
中科院分区：
文献类型：
--
作者：
Tianjun Zhang;Z. Yao;A. Gholami;Joseph Gonzalez;K. Keutzer;Michael W. Mahoney;G. Biros
已经观察到，残留网络可以看作是普通微分方程（ODE）的显式欧拉离散化。这一观察结果促使引入了所谓的神经ODE，其中其他离散化方案和/或自适应时间踏脚技术可用于改善残留网络的性能。在这里，我们提出了\ outs，它通过引入一个框架来扩展这种方法，该框架以耦合公式允许基于权重和激活的基于ODE的演变。这种方法提供了更多的建模灵活性，并且可以帮助概括性能。我们介绍了\我们的表述，得出最佳条件，并在Pytorch中实现了耦合框架。我们使用\我们的几种不同配置提出了经验结果，并在CIFAR-10数据集上对其进行了测试。我们报告的结果表明，与基线重新网络网络和最近所提供的神经ODE方法相比，我们基于耦合的框架确实是可以训练的，并且可以达到更高的精度。
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch. We present empirical results using several different configurations of \OURS, testing them on the CIFAR-10 dataset. We report results showing that our coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, compared to the baseline ResNet network and the recently-proposed Neural ODE approach.