Language-Agnostic Compiler Testing

与语言无关的编译器测试

• 批准号: RGPIN-2019-06106
• 负责人: Sun, Chengnian
• 金额: $ 2.04万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=715930
• 关键词: Language; Agnostic; Compiler; Testing

Compilers are among the most important, widely-used and complex software systems ever written. They are the trusted foundation for building other software (and also themselves). Perhaps less known to application developers is that production compilers do also contain bugs (in fact quite a few), which frustrate programmers and may lead to mysterious program failures or even catastrophic consequences in safety-critical domains. For example, compiler crashes hinder software development and maintenance process, and bugs in optimizers are daunting because they cause programs silently to be compiled to wrong binary code. To prevent these nuisances, it is important to devise novel, effective methodologies, techniques and tools for validating complex, real-world compilers. As formal compiler verification is yet to be practical for production compilers, testing remains the dominant technique for compiler quality assurance. However, testing is inadequate in general because of the complexity of compilers and the difficulty in constructing rich sets of test programs. The key challenge of compiler testing is how to effectively generate diverse test programs to exercise different code paths in compilers. Moreover, a large number of programming languages exist. A good testing technique is usually specific to a single language. To apply a testing technique to a compiler of another language, a substantial amount of engineering effort is usually required to re-implement/customize the technique. This proposal aims to explore novel, practical techniques for validating multiple compilers of different languages. Specifically, the goal is to devise a general approach to automatically generate diverse, high-quality test programs. Concretely, I plan to tackle this problem in three different ways: (1) language-agnostic program generation with constraint solving, (2) language-agnostic program mutation, and (3) program generation with machine learning. All the three directions are orthogonal to, and complement one another. This program concerns a technically very challenging problem and can lead to novel techniques for validating multiple production compilers. The program can also provide deeper and more thorough understanding of the characteristics of compiler bugs. The gained insights can help advance the state-of-the-art in engineering more trustworthy and user-friendly compilers. First, developers will benefit from this program as compilers are indispensable components of the software development process. Second, the program will not only lead to more reliable and usable compilers, but also indirectly improve the quality of every piece of software that our daily lives increasingly depend upon. Third, this program will also benefit local Canadian companies that focus on developing compilers for different platforms, e.g., FPGA compilers.

编译器是有史以来最重要、最广泛使用和最复杂的软件系统之一。它们是构建其他软件（以及它们自己）的可信基础。应用程序开发人员可能不太了解的是，生产编译器也包含错误（实际上相当多），这些错误会让程序员感到沮丧，并可能导致神秘的程序故障，甚至在安全关键领域中造成灾难性后果。例如，编译器崩溃阻碍了软件开发和维护过程，优化器中的错误令人生畏，因为它们会导致程序被无声地编译为错误的二进制代码。为了防止这些麻烦，重要的是要设计新的，有效的方法，技术和工具来验证复杂的，现实世界的编译器。 由于正式的编译器验证对于生产编译器还不实用，测试仍然是编译器质量保证的主要技术。然而，由于编译器的复杂性和构造丰富的测试程序集的困难，测试通常是不够的。编译器测试的关键挑战是如何有效地生成不同的测试程序来执行编译器中的不同代码路径。此外，存在大量的编程语言。一个好的测试技术通常是特定于一种语言的。为了将测试技术应用于另一种语言的编译器，通常需要大量的工程工作来重新实现/定制该技术。 这个建议的目的是探索新的，实用的技术来验证不同语言的多个编译器。具体来说，我们的目标是设计一个通用的方法来自动生成多样化的，高质量的测试程序。具体地说，我计划用三种不同的方法来解决这个问题：（1）与约束求解无关的语言程序生成，（2）与语言无关的程序变异，以及（3）与机器学习无关的程序生成。这三个方向都是正交的，并且相互补充。 这个程序涉及一个技术上非常具有挑战性的问题，并可能导致新的技术来验证多个生产编译器。该程序还可以提供对编译器错误特征的更深入和更透彻的理解。所获得的见解可以帮助推进最先进的工程更值得信赖和用户友好的编译器。首先，开发人员将受益于此程序，因为编译器是软件开发过程中不可或缺的组件。其次，该程序不仅会带来更可靠和可用的编译器，而且会间接提高我们日常生活日益依赖的每一个软件的质量。第三，该计划还将使专注于为不同平台开发编译器的加拿大本地公司受益，例如，FPGA编译器。