Leveraging Test Suites for Better Program Analysis

最先进的程序理解工具包括静态和动态程序分析，它们是预测和推理程序行为的强大工具。开发人员使用静态和动态程序分析来改进他们的软件。例如，程序分析可以定位不想要的行为，并帮助强制执行所需的属性，包括安全属性；它还可以在程序开发期间回答开发人员的问题。任何程序分析，无论是静态的还是动态的，都可以从关于程序的额外信息中受益。现代编程方法期望开发人员与他们的软件一起维护自动化测试套件。测试套件编码开发人员对正确程序行为的理解。请考虑以下类比。一开始，一盘棋有很多种可能性。每一步都限制了玩家可用的可能性-结束游戏足够简单，即使是初学者也能分析。同样，每个测试用例都将注意力集中在程序的特定部分上。限制问题范围使分析能够对该范围内的程序行为做出强有力的保证。建议研究背后的关键思想是利用测试套件来实现更有效的程序分析。拟议研究方案的总体长期目标是发展一种静态和动态分析相结合的理论。这一理论将支持程序分析的设计，这些程序分析集成了来自源代码(如今天所做的)及其测试用例(这是新的)的信息。新颖的混合分析将利用静态分析和动态分析的优势，使用测试套件来驱动动态分析并为静态分析提供信息。动态分析可以帮助改进静态分析固有的近似值，并减少报告的误报数量。静态分析可以扩大从动态分析获得的结果的适用性。例如，该理论的具体应用包括自动检测测试套件中的不一致。关键的见解是，测试套件可以提供关于预期程序行为的额外信息，这一见解可以很容易地纳入其他人的研究计划，有助于拟议的研究的学术影响。业界对改进测试实践有广泛的兴趣。开发人员编写了大量的测试代码，并在软件维护上花费了大量的精力，这一点得到了测试套件的关键支持。提高测试套件质量和可用性的结果可以迅速被加拿大工业界采用。

State-of-the-art tools for understanding programs include static and dynamic program analyses, which are powerful tools for predicting and reasoning about program behaviours. Developers use static and dynamic program analysis to improve their software. For example, program analysis can locate undesirable behaviours and help enforce desired properties, including security properties; and it can answer developer questions during program development. Any program analysis, whether static or dynamic, could benefit from additional information about the program.Modern programming methodologies expect developers to maintain automated test suites along with their software. Test suites encode developer understanding of correct program behaviours.Consider the following analogy. At its outset, a game of chess allows a vast number of possibilities. Each move restricts the available possibilities for the players---endgames are simple enough for even beginners to analyze. Similarly, each test case focusses attention on specific parts of the program. Restricting the problem scope enables analyses to make strong guarantees about program behaviour within that scope.The key idea behind the proposed research is to leverage test suites to enable more effective program analyses. The overall long-term objective of the proposed research programme is to develop a theory of combined static and dynamic analysis. This theory will support the design of program analyses that integrate information from source code (as done today) and its test cases (which is novel). Novel hybrid analyses will leverage the strengths of both static and dynamic analyses, using test suites to drive the dynamic analysis and provide information for the static analysis. Dynamic analysis can help refine the approximations inherent to static analysis and reduce the number of reported false positives. Static analysis can broaden the applicability of the results obtained from dynamic analysis.Specific applications of this theory include, for instance, automatically detecting inconsistencies in test suites. The key insight, that test suites can provide additional information about expected program behaviours, is one that can easily be incorporated into others' research programmes, contributing to the proposed research's academic impact.There is widespread industry interest in improving test practices. Developers write enormous amounts of test code, and expend substantial effort on software maintenance, which is critically supported by test suites. Results that improve the quality and usability of test suites can be quickly adopted by Canadian industry.