SHF: Small: Science and Tools for Intelligent Developer Testing

SHF：小型：智能开发人员测试的科学和工具

• 批准号: 1816615
• 负责人: Tianyin Xu
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816615&HistoricalAwards=false
• 关键词: SHF; Small; Science; Tools; Intelligent

Software dependability plays a critical role in United States businesses, government, and society. Although much progress has been made in software verification and validation, software testing remains by far the most widely used technique for improving software dependability. Among various types of testing, developer testing has been widely recognized as an important and valuable means of improving software dependability. The popularity and benefits of developer testing have been well witnessed in industry; however, manual developer testing is known to be labor intensive, and is often insufficient in comprehensively exercising behavior of the software under test to expose its hidden bugs. To address the issue, one of the common ways is to use testing infrastructures and tools to reduce or complement manual testing effort to achieve higher software dependability. In the past decade, the software testing research community has made significant progress in automatic test generation. With various recent scientific advances by the research community, a question naturally arises: what would be the audacious goal for the field of developer testing in the upcoming decade to bring a much higher testing effectiveness and efficiency to developers? To address this question, this project investigates the science and tools of intelligent developer testing, fundamentally advancing knowledge and understanding in foundations, techniques, and tools for intelligent developer testing. The project improves software dependability by revealing more bugs during software development before they manifest in deployed software.The project focuses on science and tools for instilling intelligence from two major ways (natural language interfacing and continuous learning) into developer testing tools as part of the efforts for realizing the vision of intelligent software engineering. The project develops novel and practical techniques and tools of intelligent developer testing that have high potential to impact the industry. In particular, the project focuses on parameterized unit tests, which are an improvement over conventional unit tests because they can easily be extended by automated tools to increase code coverage while reusing developer-written oracles. The PI plans to develop intelligent tools that will make it significantly easier for developers to write parameterized unit tests with the goal of automatically translating developer intents into parts of executable test cases. This project advances the science by exploring a series of questions, e.g., how to define or determine levels of intelligence in developer testing, how to bring high levels of intelligence in developer testing tools, how to synergistically integrate machine intelligence and human intelligence (e.g., domain knowledge or insight) to effectively tackle challenging tasks in developer testing. The project involves research collaborations with industrial partners and involves participation of students from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

软件可靠性在美国企业、政府和社会中起着至关重要的作用。尽管在软件验证和确认方面已经取得了很大的进展，但软件测试仍然是迄今为止用于提高软件可靠性的最广泛使用的技术。在各种类型的测试中，开发人员测试已被广泛认为是提高软件可靠性的重要且有价值的手段。开发人员测试的普及和好处已经在行业中得到了很好的证明;然而，手动开发人员测试被认为是劳动密集型的，并且通常不足以全面地锻炼被测软件的行为以暴露其隐藏的错误。为了解决这个问题，常用的方法之一是使用测试基础设施和工具来减少或补充手动测试工作，以实现更高的软件可靠性。在过去的十年中，软件测试研究界在自动测试生成方面取得了重大进展。随着研究界最近的各种科学进步，一个问题自然出现了：在未来十年中，开发人员测试领域的大胆目标是什么，以便为开发人员带来更高的测试效率和效率？为了解决这个问题，该项目研究了智能开发人员测试的科学和工具，从根本上推进了智能开发人员测试的基础，技术和工具的知识和理解。该项目通过在软件开发过程中发现更多的错误，从而提高软件的可靠性。该项目侧重于从两个主要途径（自然语言接口和持续学习）向开发人员测试工具灌输智能的科学和工具，作为实现智能软件工程愿景的努力的一部分。该项目开发了新颖实用的智能开发人员测试技术和工具，具有很大的影响行业的潜力。特别是，该项目专注于参数化单元测试，这是对传统单元测试的改进，因为它们可以很容易地通过自动化工具进行扩展，以增加代码覆盖率，同时重用开发人员编写的Oracle。PI计划开发智能工具，使开发人员更容易编写参数化单元测试，目标是将开发人员的意图自动转换为可执行测试用例的一部分。该项目通过探索一系列问题来推进科学，例如，如何定义或确定开发者测试中的智能水平，如何在开发者测试工具中带来高水平的智能，如何协同地集成机器智能和人类智能（例如，领域知识或洞察力），以有效地解决开发人员测试中的挑战性任务。该项目涉及与工业合作伙伴的研究合作，并涉及来自代表性不足群体的学生的参与。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Probabilistic and Systematic Coverage of Consecutive Test-Method Pairs for Detecting Order-Dependent Flaky Tests

• DOI: 10.1007/978-3-030-72016-2_15
• 发表时间: 2021-03-01
• 期刊: Tools and Algorithms for the Construction and Analysis of Systems
• 作者: Wei A;Yi P;Xie T;Marinov D;Lam W
• 通讯作者: Lam W

SemRegex: A Semantics-Based Approach for Generating Regular Expressions from Natural Language Specifications

• DOI: 10.18653/v1/d18-1189
• 发表时间: 2018-10
• 作者: Zexuan Zhong;Jiaqi Guo;Wei Yang;Jian Peng;Tao Xie;Jian-Guang Lou;Ting Liu;D. Zhang

Reasoning about modern datacenter infrastructures using partial histories

• DOI: 10.1145/3458336.3465276
• 发表时间: 2021-06
• 期刊: Proceedings of the Workshop on Hot Topics in Operating Systems
• 作者: Xudong Sun;L. Suresh;Aishwarya Ganesan;Ramnatthan Alagappan;Michael Gasch;Lilia Tang;Tianyin Xu

Quality Assessment for Large-Scale Industrial Software Systems: Experience Report at Alibaba

大规模工业软件系统的质量评估：阿里巴巴的经验报告

• DOI: 10.1109/apsec48747.2019.00028
• 发表时间: 2019
• 期刊: Quality Assessment for Large-Scale Industrial Software Systems: Experience Report at Alibaba
• 作者: Zhi, Chen;Deng, Shuiguang;Yin, Jianwei;Fu, Min;Zhu, Hai;Li, Yuanping;Xie, Tao
• 通讯作者: Xie, Tao

FACTS: automated black-box testing of FinTech systems

• DOI: 10.1145/3236024.3275533
• 发表时间: 2018-10
• 期刊: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: Qingshun Wang;Lintao Gu;Minhui Xue;Lihua Xu;Wenyu Niu;Liang Dou;Liang He;Tao Xie