Liquid Time-constant Networks
Liquid Time-constant Networks
复制标题
DOI：
10.1609/aaai.v35i9.16936
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Ramin M. Hasani;Mathias Lechner;Alexander Amini;D. Rus;R. Grosu
中科院分区：
文献类型：
--
作者：
Ramin M. Hasani;Mathias Lechner;Alexander Amini;D. Rus;R. Grosu
我们引入了一类新的时间连续递归神经网络模型。我们不是用隐含的非线性来声明学习系统的动力学，而是构造由非线性互联门调制的一阶线性动力系统的网络。所得到的模型表示具有与其隐藏状态耦合的变化(即，液体)时间常数的动态系统，其输出由数值微分方程解算器计算。这些神经网络表现出稳定和有界的行为，在神经常微分方程组中产生了优越的表达能力，并提高了时间序列预测任务的性能。为了证明这些性质，我们首先采用理论方法找到它们的动力学边界，并通过在潜在轨迹空间中的轨迹长度度量来计算它们的表达能力。然后，我们进行了一系列的时间序列预测实验，以验证液体时间常数网络(LTCs)相对于经典和现代RNN的逼近能力。
We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics, and compute their expressive power by the trajectory length measure in a latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of Liquid Time-Constant Networks (LTCs) compared to classical and modern RNNs.