Collaborative Research: SHF: Medium: Natural Language Models with Execution Data for Software Testing

协作研究：SHF：媒介：用于软件测试的具有执行数据的自然语言模型

• 批准号: 2313027
• 负责人: Milos Gligoric
• 金额: $ 90万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313027&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Natural

Natural Language Processing (NLP) models have proven useful for various software engineering tasks, including code completion, comment generation and update, code review generation, and clone detection. Despite the importance of software testing in industry, there has been little work on using these Artificial Intelligence (AI) models for developing and maintaining test code, which is a key part of software testing in the real world. Test code differs in multiple ways from regular code: (1) Test code is structured in a specific way, with steps for setting up a test environment and comparing expected results; (2) Test code has richer context, such as the specific methods and code it is testing (code under test); (3) Test code uses different code elements than the code under test, i.e., it has a different control structure; (4) Test code has specific input values and expected results; (5) Unlike regular code, test code can be readily executed.The goal of this project is to increase the productivity of software engineers via NLP models that simplify the development and maintenance of tests (NLP4Test). Specifically, tasks include test generation and completion, test update (when the underlying code changes), and automatically migrating tests across different programming languages. This project explores testing both general codebases and emerging machine learning (ML) applications. The project targets a novel domain -- NLP4Test, and this domain requires innovative NLP models. The outcome of this project will include novel techniques, implementations of these techniques, and extensive evaluations on open-source projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自然语言处理（NLP）模型已被证明对各种软件工程任务非常有用，包括代码完成，注释生成和更新，代码审查生成和克隆检测。尽管软件测试在工业中很重要，但很少有人使用这些人工智能（AI）模型来开发和维护测试代码，这是真实的世界中软件测试的关键部分。 测试代码在多个方面与常规代码不同：（1）测试代码以特定方式结构化，具有用于设置测试环境和比较预期结果的步骤;（2）测试代码具有更丰富的上下文，例如它正在测试的特定方法和代码（测试中的代码）;（3）测试代码使用与测试中的代码不同的代码元素，即，它具有不同的控制结构;（4）测试代码有特定的输入值和预期的结果;（5）与常规代码不同，测试代码可以很容易地执行。具体来说，任务包括测试生成和完成、测试更新（当底层代码更改时）以及跨不同编程语言自动迁移测试。该项目探索测试通用代码库和新兴的机器学习（ML）应用程序。该项目针对一个新的领域-NLP 4 Test，该领域需要创新的NLP模型。该项目的成果将包括新技术、这些技术的实现以及对开源项目的广泛评估。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

项目成果

Multilingual Code Co-evolution using Large Language Models

• DOI: 10.1145/3611643.3616350
• 发表时间: 2023-07
• 期刊: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: Jiyang Zhang;Pengyu Nie;Junyi Jessy Li;Miloš Gligorić

JOG: Java JIT Peephole Optimizations and Tests from Patterns

JOG：Java JIT 窥孔优化和模式测试

• 发表时间: 2024
• 期刊: Tool Demonstrations Track
• 作者: Zang, Zhiqiang;Thimmaiah, Aditya;Gligoric, Milos
• 通讯作者: Gligoric, Milos