SHF: Small: Science and Tools for Intelligent Developer Testing

SHF：小型：智能开发人员测试的科学和工具

• 批准号: 1816615
• 负责人: Tianyin Xu
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816615&HistoricalAwards=false
• 关键词: SHF; Small; Science; Tools; Intelligent

Software dependability plays a critical role in United States businesses, government, and society. Although much progress has been made in software verification and validation, software testing remains by far the most widely used technique for improving software dependability. Among various types of testing, developer testing has been widely recognized as an important and valuable means of improving software dependability. The popularity and benefits of developer testing have been well witnessed in industry; however, manual developer testing is known to be labor intensive, and is often insufficient in comprehensively exercising behavior of the software under test to expose its hidden bugs. To address the issue, one of the common ways is to use testing infrastructures and tools to reduce or complement manual testing effort to achieve higher software dependability. In the past decade, the software testing research community has made significant progress in automatic test generation. With various recent scientific advances by the research community, a question naturally arises: what would be the audacious goal for the field of developer testing in the upcoming decade to bring a much higher testing effectiveness and efficiency to developers? To address this question, this project investigates the science and tools of intelligent developer testing, fundamentally advancing knowledge and understanding in foundations, techniques, and tools for intelligent developer testing. The project improves software dependability by revealing more bugs during software development before they manifest in deployed software.The project focuses on science and tools for instilling intelligence from two major ways (natural language interfacing and continuous learning) into developer testing tools as part of the efforts for realizing the vision of intelligent software engineering. The project develops novel and practical techniques and tools of intelligent developer testing that have high potential to impact the industry. In particular, the project focuses on parameterized unit tests, which are an improvement over conventional unit tests because they can easily be extended by automated tools to increase code coverage while reusing developer-written oracles. The PI plans to develop intelligent tools that will make it significantly easier for developers to write parameterized unit tests with the goal of automatically translating developer intents into parts of executable test cases. This project advances the science by exploring a series of questions, e.g., how to define or determine levels of intelligence in developer testing, how to bring high levels of intelligence in developer testing tools, how to synergistically integrate machine intelligence and human intelligence (e.g., domain knowledge or insight) to effectively tackle challenging tasks in developer testing. The project involves research collaborations with industrial partners and involves participation of students from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

软件可靠性在美国企业，政府和社会中起着至关重要的作用。尽管在软件验证和验证方面取得了很多进展，但迄今为止，软件测试仍然是提高软件可靠性的最广泛使用的技术。在各种类型的测试中，开发人员测试已被广泛认为是改善软件可靠性的重要且有价值的手段。开发人员测试的受欢迎程度和好处在行业中得到了很好的见证。但是，已知手动开发人员测试是劳动量大的，并且通常不足以全面地行使正在测试的软件的行为以暴露其隐藏的错误。为了解决这个问题，一种常见方法之一是使用测试基础架构和工具来减少或补充手动测试工作，以实现更高的软件可靠性。 In the past decade, the software testing research community has made significant progress in automatic test generation.随着研究界最近的各种科学进步，自然而然地出现了一个问题：在接下来的十年中，开发人员测试领域的大胆目标是什么，从而为开发人员带来了更高的测试效率和效率？为了解决这个问题，该项目研究了智能开发人员测试的科学和工具，从根本上推进了智能开发人员测试的基础，技术和工具中的知识和理解。该项目通过在软件开发中揭示更多的错误在已部署软件中揭示了更多的错误来提高软件的可靠性。该项目着重于科学和工具，用于从两种主要方式（自然语言接口和持续学习）灌输智能中的工具中，以发展开发人员测试工具，作为实现智能软件工程愿景的努力的一部分。该项目开发了智能开发人员测试的新颖和实用技术和工具，这些技术具有很高的影响行业的潜力。特别是，该项目着重于参数化的单元测试，这是对常规单元测试的改进，因为它们可以通过自动化工具轻松扩展，以增加代码覆盖范围，同时重复使用开发人员编写的Oracles。 PI计划开发智能工具，这将使开发人员更容易编写参数化的单元测试，以自动将开发人员的意图转化为可执行的测试用例的一部分。该项目通过探索一系列问题，例如，如何定义或确定开发人员测试中的智能水平，如何在开发人员测试工具中带来高水平的智能，如何协同使机器智能和人类智能（例如，领域知识或洞察力）有效地解决开发人员测试中的挑战任务。该项目涉及与工业合作伙伴的研究合作，并涉及来自代表性不足的小组的学生的参与。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的影响评估标准，被认为值得通过评估来提供支持。

项目成果

SemRegex: A Semantics-Based Approach for Generating Regular Expressions from Natural Language Specifications

• DOI: 10.18653/v1/d18-1189
• 发表时间: 2018-10
• 作者: Zexuan Zhong;Jiaqi Guo;Wei Yang;Jian Peng;Tao Xie;Jian-Guang Lou;Ting Liu;D. Zhang

Probabilistic and Systematic Coverage of Consecutive Test-Method Pairs for Detecting Order-Dependent Flaky Tests

• DOI: 10.1007/978-3-030-72016-2_15
• 发表时间: 2021-03-01
• 期刊: Tools and Algorithms for the Construction and Analysis of Systems
• 作者: Wei A;Yi P;Xie T;Marinov D;Lam W
• 通讯作者: Lam W

Quality Assessment for Large-Scale Industrial Software Systems: Experience Report at Alibaba

大规模工业软件系统的质量评估：阿里巴巴的经验报告

• DOI: 10.1109/apsec48747.2019.00028
• 发表时间: 2019
• 期刊: Quality Assessment for Large-Scale Industrial Software Systems: Experience Report at Alibaba
• 作者: Zhi, Chen;Deng, Shuiguang;Yin, Jianwei;Fu, Min;Zhu, Hai;Li, Yuanping;Xie, Tao
• 通讯作者: Xie, Tao

Reasoning about modern datacenter infrastructures using partial histories

• DOI: 10.1145/3458336.3465276
• 发表时间: 2021-06
• 期刊: Proceedings of the Workshop on Hot Topics in Operating Systems
• 作者: Xudong Sun;L. Suresh;Aishwarya Ganesan;Ramnatthan Alagappan;Michael Gasch;Lilia Tang;Tianyin Xu

FACTS: automated black-box testing of FinTech systems

• DOI: 10.1145/3236024.3275533
• 发表时间: 2018-10
• 期刊: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: Qingshun Wang;Lintao Gu;Minhui Xue;Lihua Xu;Wenyu Niu;Liang Dou;Liang He;Tao Xie