Collaborative Research: SHF: Medium: Natural Language Models with Execution Data for Software Testing

协作研究：SHF：媒介：用于软件测试的具有执行数据的自然语言模型

• 批准号: 2313027
• 负责人: Milos Gligoric
• 金额: $ 90万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313027&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Natural

Natural Language Processing (NLP) models have proven useful for various software engineering tasks, including code completion, comment generation and update, code review generation, and clone detection. Despite the importance of software testing in industry, there has been little work on using these Artificial Intelligence (AI) models for developing and maintaining test code, which is a key part of software testing in the real world. Test code differs in multiple ways from regular code: (1) Test code is structured in a specific way, with steps for setting up a test environment and comparing expected results; (2) Test code has richer context, such as the specific methods and code it is testing (code under test); (3) Test code uses different code elements than the code under test, i.e., it has a different control structure; (4) Test code has specific input values and expected results; (5) Unlike regular code, test code can be readily executed.The goal of this project is to increase the productivity of software engineers via NLP models that simplify the development and maintenance of tests (NLP4Test). Specifically, tasks include test generation and completion, test update (when the underlying code changes), and automatically migrating tests across different programming languages. This project explores testing both general codebases and emerging machine learning (ML) applications. The project targets a novel domain -- NLP4Test, and this domain requires innovative NLP models. The outcome of this project will include novel techniques, implementations of these techniques, and extensive evaluations on open-source projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自然语言处理（NLP）模型已被证明对各种软件工程任务有用，包括代码完成，评论生成和更新，代码审查生成和克隆检测。尽管软件测试在行业中的重要性，但使用这些人工智能（AI）模型来开发和维护测试代码的工作很少，这是现实世界中软件测试的关键部分。 测试代码以多种方式不同与常规代码不同：（1）测试代码以特定方式构成，并进行设置测试环境并比较预期结果的步骤； （2）测试代码具有更丰富的上下文，例如它正在测试的特定方法和代码（正在测试的代码）； （3）测试代码使用与正在测试的代码不同的代码元素，即它具有不同的控制结构； （4）测试代码具有特定的输入值和预期结果； （5）与常规代码不同，可以容易执行测试代码。该项目的目的是通过NLP模型来提高软件工程师的生产率，从而简化测试的开发和维护（NLP4TEST）。具体而言，任务包括测试生成和完成，测试更新（基础代码更改时）以及自动跨不同编程语言迁移测试。该项目探讨了一般代码库和新兴机器学习（ML）应用程序的测试。该项目针对一个新的域 - NLP4Test，该域需要创新的NLP模型。该项目的结果将包括新技术，这些技术的实施以及对开源项目的广泛评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估评估标准的评估值得支持的。

项目成果

JOG: Java JIT Peephole Optimizations and Tests from Patterns

JOG：Java JIT 窥孔优化和模式测试

• 发表时间: 2024
• 期刊: Tool Demonstrations Track
• 作者: Zang, Zhiqiang;Thimmaiah, Aditya;Gligoric, Milos
• 通讯作者: Gligoric, Milos

Multilingual Code Co-evolution using Large Language Models

• DOI: 10.1145/3611643.3616350
• 发表时间: 2023-07
• 期刊: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: Jiyang Zhang;Pengyu Nie;Junyi Jessy Li;Miloš Gligorić