On Second Order Behaviour in Augmented Neural ODEs
On Second Order Behaviour in Augmented Neural ODEs
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Alexander Norcliffe;Cristian Bodnar;Ben Day;Nikola Simidjievski;P. Lio’
中科院分区：
文献类型：
--
作者：
Alexander Norcliffe;Cristian Bodnar;Ben Day;Nikola Simidjievski;P. Lio’
神经常微分方程（NODE）是一类新的模型，通过无限深度架构连续转换数据。NODE的连续性使其特别适合于学习复杂物理系统的动力学。虽然以前的工作主要集中在一阶常微分方程上，但许多系统的动力学，特别是经典物理中的系统，都是由二阶定律控制的。在这项工作中，我们考虑二阶神经常微分方程（SONODE）。我们展示了如何伴随灵敏度方法可以扩展到SONODE，并证明一阶耦合ODE的优化是等价的，计算效率更高。此外，我们扩展了对更广泛的增强节点（ANODE）的理论理解，表明它们也可以用最小数量的增强维度学习高阶动力学，但代价是可解释性。这表明ANODE的优势超出了最初认为的增加维度所提供的额外空间。最后，我们比较了SONODE和ANODE在合成和真实的动力系统上的表现，并证明了前者的归纳偏差通常会导致更快的训练和更好的性能。
Neural Ordinary Differential Equations (NODEs) are a new class of models that transform data continuously through infinite-depth architectures. The continuous nature of NODEs has made them particularly suitable for learning the dynamics of complex physical systems. While previous work has mostly been focused on first order ODEs, the dynamics of many systems, especially in classical physics, are governed by second order laws. In this work, we consider Second Order Neural ODEs (SONODEs). We show how the adjoint sensitivity method can be extended to SONODEs and prove that the optimisation of a first order coupled ODE is equivalent and computationally more efficient. Furthermore, we extend the theoretical understanding of the broader class of Augmented NODEs (ANODEs) by showing they can also learn higher order dynamics with a minimal number of augmented dimensions, but at the cost of interpretability. This indicates that the advantages of ANODEs go beyond the extra space offered by the augmented dimensions, as originally thought. Finally, we compare SONODEs and ANODEs on synthetic and real dynamical systems and demonstrate that the inductive biases of the former generally result in faster training and better performance.