Language-Agnostic Compiler Testing

编译器是有史以来最重要、最广泛使用和最复杂的软件系统之一。它们是构建其他软件（也包括它们自己）的可信基础。应用程序开发人员可能不太了解的是，产品编译器也包含错误（实际上相当多），这使程序员感到沮丧，并可能导致神秘的程序失败，甚至在安全关键领域造成灾难性的后果。例如，编译器崩溃会阻碍软件开发和维护过程，优化器中的错误令人生畏，因为它们会导致程序被悄悄地编译成错误的二进制代码。为了防止这些麻烦，设计新颖、有效的方法、技术和工具来验证复杂的、真实的编译器是很重要的。由于正式的编译器验证对于生产编译器还不实用，测试仍然是编译器质量保证的主要技术。然而，由于编译器的复杂性和构建丰富的测试程序集的困难，测试通常是不充分的。编译器测试的关键挑战是如何有效地生成不同的测试程序来在编译器中运行不同的代码路径。此外，存在大量的编程语言。好的测试技术通常是针对一种语言的。要将测试技术应用于另一种语言的编译器，通常需要大量的工程工作来重新实现/定制该技术。本提案旨在探索验证不同语言的多种编译器的新颖实用技术。具体地说，目标是设计一种通用的方法来自动生成不同的、高质量的测试程序。具体来说，我计划用三种不同的方式来解决这个问题：(1)使用约束求解的语言无关程序生成，(2)使用语言无关程序突变，以及(3)使用机器学习的程序生成。这三个方向互相正交，互为补。这个程序涉及一个技术上非常具有挑战性的问题，并可能导致验证多个生产编译器的新技术。该程序还可以对编译器错误的特征提供更深入、更彻底的理解。获得的见解可以帮助提高工程中的最先进技术，使编译器更加值得信赖和用户友好。首先，开发人员将从这个程序中受益，因为编译器是软件开发过程中不可或缺的组件。其次，该程序不仅会导致更可靠和可用的编译器，而且还会间接提高我们日常生活日益依赖的每一个软件的质量。第三，该项目还将使加拿大本地专注于开发不同平台编译器的公司受益，例如FPGA编译器。

Compilers are among the most important, widely-used and complex software systems ever written. They are the trusted foundation for building other software (and also themselves). Perhaps less known to application developers is that production compilers do also contain bugs (in fact quite a few), which frustrate programmers and may lead to mysterious program failures or even catastrophic consequences in safety-critical domains. For example, compiler crashes hinder software development and maintenance process, and bugs in optimizers are daunting because they cause programs silently to be compiled to wrong binary code. To prevent these nuisances, it is important to devise novel, effective methodologies, techniques and tools for validating complex, real-world compilers. As formal compiler verification is yet to be practical for production compilers, testing remains the dominant technique for compiler quality assurance. However, testing is inadequate in general because of the complexity of compilers and the difficulty in constructing rich sets of test programs. The key challenge of compiler testing is how to effectively generate diverse test programs to exercise different code paths in compilers. Moreover, a large number of programming languages exist. A good testing technique is usually specific to a single language. To apply a testing technique to a compiler of another language, a substantial amount of engineering effort is usually required to re-implement/customize the technique. This proposal aims to explore novel, practical techniques for validating multiple compilers of different languages. Specifically, the goal is to devise a general approach to automatically generate diverse, high-quality test programs. Concretely, I plan to tackle this problem in three different ways: (1) language-agnostic program generation with constraint solving, (2) language-agnostic program mutation, and (3) program generation with machine learning. All the three directions are orthogonal to, and complement one another. This program concerns a technically very challenging problem and can lead to novel techniques for validating multiple production compilers. The program can also provide deeper and more thorough understanding of the characteristics of compiler bugs. The gained insights can help advance the state-of-the-art in engineering more trustworthy and user-friendly compilers. First, developers will benefit from this program as compilers are indispensable components of the software development process. Second, the program will not only lead to more reliable and usable compilers, but also indirectly improve the quality of every piece of software that our daily lives increasingly depend upon. Third, this program will also benefit local Canadian companies that focus on developing compilers for different platforms, e.g., FPGA compilers.