Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers
Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers
复制标题
DOI：
10.1109/cdc45484.2021.9683530
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Y. Ebihara;Hayato Waki;Victor Magron;N. Mai;D. Peaucelle;S. Tarbouriech
中科院分区：
文献类型：
--
作者：
Y. Ebihara;Hayato Waki;Victor Magron;N. Mai;D. Peaucelle;S. Tarbouriech
利用积分二次约束(IQC)框架对递归神经网络(RNN)进行稳定性分析。整流线性单元(RELU)通常被用作RNN的激活函数，并且RELU具有关于其输入和输出信号的特定的非负性。因此，如果我们能够推导出基于IQC的稳定性条件，并且乘子能够照顾到这样的非负性，那么它是有效的。然而，这种非负性(线性)性质很难被定义在半正定锥上的现有乘子所捕获。为了克服这一困难，我们将标准的半正定锥松弛为余正锥，并利用余正乘子来捕捉非负性。我们证明，在IQC的框架内，我们可以使用余正乘子(或其内近似)和现有的乘子，如Zames-Falb乘子和多面体边界乘子，这直接使我们能够确保余正乘子的引入导致更好的(不再保守的)结果。最后，通过数值算例说明了基于余正乘子的IQC稳定性条件的有效性。
This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the activation function of the RNN, and the ReLU has specific nonnegativity properties regarding its input and output signals. Therefore, it is effective if we can derive IQC-based stability conditions with multipliers taking care of such nonnegativity properties. However, such nonnegativity (linear) properties are hardly captured by the existing multipliers defined on the positive semidefinite cone. To get around this difficulty, we loosen the standard positive semidefinite cone to the copositive cone, and employ copositive multipliers to capture the nonnegativity properties. We show that, within the framework of the IQC, we can employ copositive multipliers (or their inner approximation) together with existing multipliers such as Zames-Falb multipliers and polytopic bounding multipliers, and this directly enables us to ensure that the introduction of the copositive multipliers leads to better (no more conservative) results. We finally illustrate the effectiveness of the IQC-based stability conditions with the copositive multipliers by numerical examples.