What kinds of contracts do ML APIs need?
What kinds of contracts do ML APIs need?
复制标题
DOI：
10.1007/s10664-023-10320-z
复制
发表时间：
2023-07
影响因子：
4.1
通讯作者：
S. K. Samantha;Shibbir Ahmed;S. Imtiaz;Hridesh Rajan;G. Leavens
中科院分区：
文献类型：
--
作者：
S. K. Samantha;Shibbir Ahmed;S. Imtiaz;Hridesh Rajan;G. Leavens
最近的工作表明，机器学习（ML）程序容易出错，需要ML代码的合同。契约，如契约设计方法中的契约，帮助文档化API并帮助API用户编写正确的代码。问题是：什么样的合约可以为API用户提供最大的帮助？我们特别感兴趣的是什么类型的合约可以帮助API用户在ML管道的早期阶段捕获错误。我们描述了对Stack Overflow上四个最常讨论的ML库的帖子的实证研究：TensorFlow，Scikit-learn，Keras和PyTorch。对于这些库，我们的研究提取了413个非正式（英语）API规范。我们使用这些规范来理解以下问题。违反ML合同的根本原因和影响是什么？是否存在违反ML合同的常见模式？什么时候理解ML合同需要高级ML软件专业知识？在API级别检查合约是否有助于检测早期ML管道阶段的违规行为？我们的主要发现是，ML API最常用的契约要么是检查API的单个参数的约束，要么是检查API调用的顺序。软件工程社区可以使用现有的合约挖掘方法来挖掘这些合约，以促进对ML API的理解。我们还注意到需要结合联合收割机的行为和时间合同挖掘方法。我们报告了所需ML合同的类别，这可能有助于合同语言的设计者。
Recent work has shown that Machine Learning (ML) programs are error-prone and called for contracts for ML code. Contracts, as in the design by contract methodology, help document APIs and aid API users in writing correct code. The question is: what kinds of contracts would provide the most help to API users? We are especially interested in what kinds of contracts help API users catch errors at earlier stages in the ML pipeline. We describe an empirical study of posts onStack Overflowof the four most often-discussed ML libraries:TensorFlow,Scikit-learn,Keras, andPyTorch. For these libraries, our study extracted 413 informal (English) API specifications. We used these specifications to understand the following questions. What are the root causes and effects behind ML contract violations? Are there common patterns of ML contract violations? When does understanding ML contracts require an advanced level of ML software expertise? Could checking contracts at the API level help detect the violations in early ML pipeline stages? Our key findings are that the most commonly needed contracts for ML APIs are either checking constraints on single arguments of an API or on the order of API calls. The software engineering community could employ existing contract mining approaches to mine these contracts to promote an increased understanding of ML APIs. We also noted a need to combine behavioral and temporal contract mining approaches. We report on categories of required ML contracts, which may help designers of contract languages.