A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair
A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair
复制标题
DOI：
10.48550/arxiv.2304.08191
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Jialun Cao;Meiziniu Li;Ming Wen;S. Cheung
中科院分区：
文献类型：
--
作者：
Jialun Cao;Meiziniu Li;Ming Wen;S. Cheung
Chatgpt彻底改变了许多研究和工业领域。 Chatgpt在软件工程方面表现出很大的潜力，可以增强各种传统任务，例如程序维修，代码理解和代码生成。但是，是否适用于深度学习（DL）程序是否自动修复（APR）仍然未知。 DL程序的决策逻辑未在源代码中明确编码，但对APR提出了独特的挑战。在维修DL程序时，APR方法不仅需要以句法解析源代码，而且还需要了解代码意图。通过最好的先前工作，故障定位的性能仍然远远低于令人满意的（仅约30 \％）。因此，在本文中，我们通过提出三个研究问题来探讨Chatgpt进行DL计划维修的能力。 （1）CHATGPT可以有效地调试DL程序吗？ （2）如何通过提示来提高Chatgpt的维修绩效？ （3）以哪种方式对话有助于促进维修？最重要的是，我们对DL程序维修的及时设计有用的共同方面进行分类。另外，我们提出了各种及时的模板，以促进性能并总结Chatgpt能力的优势和缺点，例如检测不良代码气味，代码重构以及检测API滥用/贬值。
ChatGPT has revolutionized many research and industrial fields. ChatGPT has shown great potential in software engineering to boost various traditional tasks such as program repair, code understanding, and code generation. However, whether automatic program repair (APR) applies to deep learning (DL) programs is still unknown. DL programs, whose decision logic is not explicitly encoded in the source code, have posed unique challenges to APR. While to repair DL programs, an APR approach needs to not only parse the source code syntactically but also needs to understand the code intention. With the best prior work, the performance of fault localization is still far less than satisfactory (only about 30\%). Therefore, in this paper, we explore ChatGPT's capability for DL program repair by asking three research questions. (1) Can ChatGPT debug DL programs effectively? (2) How can ChatGPT's repair performance be improved by prompting? (3) In which way can dialogue help facilitate the repair? On top of that, we categorize the common aspects useful for prompt design for DL program repair. Also, we propose various prompt templates to facilitate the performance and summarize the advantages and disadvantages of ChatGPT's abilities such as detecting bad code smell, code refactoring, and detecting API misuse/deprecation.