Collaborative Research: SHF: Small: Automated Quantitative Assessment of Testing Difficulty

合作研究：SHF：小型：测试难度自动定量评估

• 批准号: 2008660
• 负责人: Tevfik Bultan
• 金额: $ 35.97万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008660&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Automated

Our society is heavily reliant on software systems running on an increasingly large number of programmable devices that surround us. Moreover, the amount of software in safety-critical systems such as cars and planes keeps increasing. Software-quality assurance is one of the most fundamental problems that we are facing in this modern computing-dominated era. One can read about dependability and security problems caused by poor-quality software in the news everyday. It is extremely crucial to develop techniques that can improve the quality of software systems before they cause disastrous consequences during operation. The most common software-quality assurance technique is software testing. Although there has been a surge of progress in automated software-testing techniques, it is hard to predict their effectiveness. Given a piece of software, there is no existing technique that can predict how challenging it will be to automatically test that piece of software. In this project the goal is to develop techniques for assessing the difficulty of automatically testing software.Existing software-complexity metrics do not provide meaningful assessments of testing difficulty. This project's goal is to develop scalable techniques that can provide a quantitative assessment of testing difficulty. In order to be scalable and practical, the method has to rely on a level of abstraction that provides efficient analysis, while preserving meaningful characteristics of program behavior that relate to testing difficulty. The approach used in this project builds on two concepts that provide a promising abstraction for quantitative assessment of testing difficulty: 1) path complexity, and 2) path selectivity. Path complexity assesses how the number of paths in a given program increases with increasing execution depth, and path selectivity assesses the difficulty of finding values that satisfy a path condition. The team of researchers working on this project will develop techniques that automatically compute path complexity and path selectivity and then combine them to obtain a quantitative measure for testing difficulty. By developing techniques that can assess software-testing difficulty, this project will enable development of more effective software-testing techniques based on better resource allocation for software-quality assurance tasks. This will lead to improvements in software quality, and reduction in software defects that cause dependability and security problems. Secondly, the research activity will help to expose graduate and undergraduate students to software-quality assurance challenges and techniques. Finally, the research activity will help to disseminate the knowledge, techniques and tools developed within the scope of this project through publishing in open literature and making available the software tools that are developed as open source.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

我们的社会在很大程度上依赖于在我们周围越来越多的可编程设备上运行的软件系统。此外，诸如汽车和飞机之类的安全系统中的软件量一直在增加。软件质量的保证是我们在这个以现代计算为主时代中面临的最根本问题之一。人们每天都可以阅读有关新闻中质量较差的软件引起的可靠性和安全问题。开发可以在软件系统造成灾难性后果之前，开发可以提高软件系统质量的技术至关重要。最常见的软件质量保证技术是软件测试。尽管自动化软件测试技术的进展激增，但很难预测其有效性。给定一项软件，没有现有技术可以预测自动测试该软件的挑战性。在该项目中，目标是开发用于评估自动测试软件难度的技术。存在软件 - 复杂度指标不能提供有意义的测试难度评估。该项目的目标是开发可扩展技术，以提供测试难度的定量评估。为了扩展和实用，该方法必须依靠一个提供有效分析的抽象水平，同时保留与测试难度有关的计划行为的有意义的特征。该项目中使用的方法建立在两个概念上，这些概念为测试难度定量评估提供了有希望的抽象：1）路径复杂性和2）路径选择性。路径复杂性评估给定程序中的路径数量如何随着执行深度的增加而增加，而路径选择性评估了找到满足路径条件的值的困难。从事该项目的研究人员团队将开发自动计算路径复杂性和路径选择性的技术，然后组合它们以获得测试难度的定量度量。通过开发可以评估软件测试难度的技术，该项目将基于针对软件质量保证任务的更好资源分配来开发更有效的软件测试技术。这将导致软件质量的改善，并减少导致可靠性和安全问题的软件缺陷。其次，研究活动将有助于使研究生和本科生面临软件质量的保证挑战和技术。最后，研究活动将有助于传播该项目范围内开发的知识，技术和工具，并通过在开放文献中发布并提供作为开放源代码开发的软件工具。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来通过评估来获得支持的。

项目成果

PREACH: A Heuristic for Probabilistic Reachability to Identify Hard to Reach Statements

• DOI: 10.1145/3510003.3510227
• 发表时间: 2022-05
• 期刊: 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE)
• 作者: Seemanta Saha;M. Downing;Tegan Brennan;T. Bultan

Rare Path Guided Fuzzing

• DOI: 10.1145/3597926.3598136
• 发表时间: 2023-07
• 期刊: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
• 作者: Seemanta Saha;Laboni Sarker;Md Shafiuzzaman;Chaofan Shou;Albert Li;Ganesh Sankaran;T. Bultan

Quantifying permissiveness of access control policies

量化访问控制策略的允许性

• DOI: 10.1145/3510003.3510233
• 发表时间: 2022
• 期刊: ICSE '22: Proceedings of the 44th International Conference on Software Engineering
• 作者: Eiers, William;Sankaran, Ganesh;Li, Albert;O'Mahony, Emily;Prince, Benjamin;Bultan, Tevfik
• 通讯作者: Bultan, Tevfik