Robust and provably monotonic networks
Robust and provably monotonic networks
复制标题
DOI：
10.1088/2632-2153/aced80
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
O. Kitouni;N. Nolte;Mike Williams
中科院分区：
文献类型：
--
作者：
O. Kitouni;N. Nolte;Mike Williams
由神经网络表示的输入和输出空间之间的映射的Lipschitz常数是用于评估模型的鲁棒性的自然度量。我们提出了一种新的方法来约束密集深度学习模型的Lipschitz常数，这种方法也可以推广到其他架构。该方法在训练过程中依赖于一个简单的权重归一化方案，确保每层的Lipschitz常数低于分析师指定的上限。然后，可以使用简单的单调剩余连接来使模型在其输入的任何子集中单调，这在领域知识决定这种依赖性的情况下很有用。例子可以在算法公平性要求中找到，或者如这里所介绍的，在CERN大型强子对撞机产生的亚原子粒子衰变的分类中找到。我们的规范化是最低限度的约束，并允许底层架构，以保持更高的表现力相比，其他技术的目的是控制Lipschitz常数的模型或确保其单调性。我们展示了如何使用该算法来训练一个强大的，强大的，和可解释的重味夸克衰变，这已被采用作为主要的数据选择算法在LHCb实时数据处理系统在当前的LHC数据采集期间称为运行3。此外，我们的算法在医学、金融和其他应用的基准测试中也取得了最先进的性能。
The Lipschitz constant of the map between the input and output space represented by a neural network is a natural metric for assessing the robustness of the model. We present a new method to constrain the Lipschitz constant of dense deep learning models that can also be generalized to other architectures. The method relies on a simple weight normalization scheme during training that ensures the Lipschitz constant of every layer is below an upper limit specified by the analyst. A simple monotonic residual connection can then be used to make the model monotonic in any subset of its inputs, which is useful in scenarios where domain knowledge dictates such dependence. Examples can be found in algorithmic fairness requirements or, as presented here, in the classification of the decays of subatomic particles produced at the CERN Large Hadron Collider. Our normalization is minimally constraining and allows the underlying architecture to maintain higher expressiveness compared to other techniques which aim to either control the Lipschitz constant of the model or ensure its monotonicity. We show how the algorithm was used to train a powerful, robust, and interpretable discriminator for heavy-flavor-quark decays, which has been adopted for use as the primary data-selection algorithm in the LHCb real-time data-processing system in the current LHC data-taking period known as Run 3. In addition, our algorithm has also achieved state-of-the-art performance on benchmarks in medicine, finance, and other applications.