Design by Contract for Deep Learning APIs
Design by Contract for Deep Learning APIs
复制标题
DOI：
10.1145/3611643.3616247
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Shibbir Ahmed;S. Imtiaz;S. K. Samantha;Breno Dantas Cruz;Hridesh Rajan
中科院分区：
文献类型：
--
作者：
Shibbir Ahmed;S. Imtiaz;S. K. Samantha;Breno Dantas Cruz;Hridesh Rajan
如今，深度学习（DL）技术越来越多地被纳入关键软件系统中。DL软件也有bug。SE最近的工作已经描述了这些错误，研究了修复模式，并提出了检测和定位策略。在这项工作中，我们介绍了一种预防措施。我们提出了DL库的契约式设计，简称DL契约，以记录DL库的属性，并为开发人员提供一种在开发过程中识别错误的机制。虽然DL Contract建立在传统的契约设计技术基础上，但我们需要应对独特的挑战。特别是，我们需要记录训练过程的属性，这些属性在DL库的功能接口上不可见。为了解决这些问题，我们引入了一些机制，允许开发人员指定模型架构、数据和训练过程的属性。我们为基于Python的DL库设计并实现了DL Contract，并使用它来记录Keras（一个著名的DL库）的属性。我们评估DL合同的有效性，运行时开销和可用性。为了评估DL合约的效用，我们专门针对训练问题和结构错误开发了15个样本合约。我们已经采用了四个经过严格审查的基准测试，这些基准测试来自于之前关于DL错误检测和修复的工作。对于有效性，DL合同正确地检测到259个错误在272个现实世界的错误程序，从良好的审查基准提供了在以前的工作DL错误检测和修复。我们发现，对于所使用的基准测试，DL合约的开销相当小。最后，为了评估可用性，我们对20名使用DL Contract查找和修复bug的参与者进行了调查。结果表明，DL合同可以很好地帮助DL应用程序开发人员调试他们的代码。
Deep Learning (DL) techniques are increasingly being incorporated in critical software systems today. DL software is buggy too. Recent work in SE has characterized these bugs, studied fix patterns, and proposed detection and localization strategies. In this work, we introduce a preventative measure. We propose design by contract for DL libraries, DL Contract for short, to document the properties of DL libraries and provide developers with a mechanism to identify bugs during development. While DL Contract builds on the traditional design by contract techniques, we need to address unique challenges. In particular, we need to document properties of the training process that are not visible at the functional interface of the DL libraries. To solve these problems, we have introduced mechanisms that allow developers to specify properties of the model architecture, data, and training process. We have designed and implemented DL Contract for Python-based DL libraries and used it to document the properties of Keras, a well-known DL library. We evaluate DL Contract in terms of effectiveness, runtime overhead, and usability. To evaluate the utility of DL Contract, we have developed 15 sample contracts specifically for training problems and structural bugs. We have adopted four well-vetted benchmarks from prior works on DL bug detection and repair. For the effectiveness, DL Contract correctly detects 259 bugs in 272 real-world buggy programs, from well-vetted benchmarks provided in prior work on DL bug detection and repair. We found that the DL Contract overhead is fairly minimal for the used benchmarks. Lastly, to evaluate the usability, we conducted a survey of twenty participants who have used DL Contract to find and fix bugs. The results reveal that DL Contract can be very helpful to DL application developers when debugging their code.