Counterexample-Guided Learning of Monotonic Neural Networks
Counterexample-Guided Learning of Monotonic Neural Networks
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Aishwarya Sivaraman;G. Farnadi;T. Millstein;Guy Van den Broeck
中科院分区：
文献类型：
--
作者：
Aishwarya Sivaraman;G. Farnadi;T. Millstein;Guy Van den Broeck
深度学习的广泛采用通常归因于其自动特征构建和最小归纳偏差。然而，在许多现实世界的任务中，学习的函数旨在满足特定于领域的约束。我们关注单调性约束，这种约束很常见，要求函数的输出随着特定输入特征值的增加而增加。我们开发了一种反例引导的技术，可以证明在预测时强制执行单调性约束。此外，我们提出了一种使用单调性作为深度学习的归纳偏差的技术。它的工作原理是在学习过程中迭代地合并单调性反例。与单调学习中的先前工作相反，我们针对通用 ReLU 神经网络，并且不进一步限制假设空间。我们已经在一个名为 COMET 的工具中实现了这些技术。对现实世界数据集的实验表明，与现有的单调学习器相比，我们的方法取得了最先进的结果，并且与不考虑单调性约束的训练模型相比，可以提高模型质量。
The widespread adoption of deep learning is often attributed to its automatic feature construction with minimal inductive bias. However, in many real-world tasks, the learned function is intended to satisfy domain-specific constraints. We focus on monotonicity constraints, which are common and require that the function's output increases with increasing values of specific input features. We develop a counterexample-guided technique to provably enforce monotonicity constraints at prediction time. Additionally, we propose a technique to use monotonicity as an inductive bias for deep learning. It works by iteratively incorporating monotonicity counterexamples in the learning process. Contrary to prior work in monotonic learning, we target general ReLU neural networks and do not further restrict the hypothesis space. We have implemented these techniques in a tool called COMET. Experiments on real-world datasets demonstrate that our approach achieves state-of-the-art results compared to existing monotonic learners, and can improve the model quality compared to those that were trained without taking monotonicity constraints into account.