CRI: CI-P: Collaborative: Towards a Program Analysis Collaboratory

基准程序在评价程序分析研究进展方面起着重要作用。因此，评估的可信度取决于基准程序的数量和质量。它们的数量越大，越接近真实世界的程序，评估的可信度就越高。目前，程序分析领域的研究往往依赖于一些过时的或有缺陷的基准程序进行评估。获得足够的基准程序的困难来自于程序分析人员对基准程序的要求。通常研究人员手动获取，检查和转换潜在的基准程序，以创建满足这些要求的基准程序。这种不可扩展的方法通常无法产生足够的基准程序。这项研究的目标是使这一过程自动化，并通过在线公共基础设施提供。该基础设施有助于获取、选择和转换开源软件项目，以用作基准程序。此外，该基础设施使研究人员能够轻松地策划用于程序分析的基准程序，并共享在这些基准上运行分析器的结果。这个项目的目标是征求要求和反馈，从程序分析社区自动基准程序生成，并原型的程序分析协作实验室（PAClab）的研究基础设施，实现这一进程。使用研究人员的规范，充分的基准程序生成，PAClab定位和获得潜在的基准程序在开源存储库;执行必要的程序转换，由研究人员指定;和输出足够的基准程序。此外，PAClab使研究人员能够使用容器技术共享他们的程序分析器，转换的基准程序以及在这些基准程序上运行分析器的结果。该项目的智力价值在于调查技术，为目标程序分析器指定和自动化程序转换。该项目的更广泛的影响源于它的潜力，以扩大适应程序分析的进步，通过增加他们的评估的信心;降低新的程序分析研究人员的准入门槛;并加快程序分析研究的步伐。PAClab的教育优势包括学生能够轻松地找到真实世界的程序和评估他们的程序分析器的实现。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Benchmark programs play an important role in evaluating the advances in program analysis research. Thus, the confidence of the evaluation depends on the quantity and the quality of the benchmark programs. The larger their number and the closer they resemble real-world programs, the higher confidence in the evaluation. Currently, research in the field of program analysis often relies on a few outdated or deficient benchmark programs for evaluation purposes. The difficulty in obtaining sufficient benchmark programs comes from the requirements imposed on the benchmark programs by program analyzers. Usually researchers manually obtain, inspect, and transform potential benchmark programs to create benchmark programs that meet those requirements. Such an unscalable approach often fails to produce adequate benchmark programs. The goal of this research is to automate this process and make it available through an online, public infrastructure. The infrastructure facilitates obtaining, selecting, and transforming open-source software projects for use as benchmark programs. In addition, the infrastructure gives researchers the ability to easily curate benchmark programs for program analysis and share the results of running their analyzers on those benchmarks. The objectives of this project are to solicit requirements and feedback from the program analysis community on automating benchmark program generation, and to prototype a Program Analysis Collaboratory (PAClab) research infrastructure that implements that process. Using researchers' specifications for adequate benchmark program generation, PAClab locates and obtains potential benchmark programs in open-source repositories; performs necessary program transformations as specified by researchers; and outputs the adequate benchmark programs. In addition, PAClab enables researchers to share their program analyzers using container technology, the transformed benchmark programs, and the results of running their analyzers on those benchmark programs. The intellectual merit of the project lies in investigating techniques for specifying and automating program transformations for targeted program analyzers. The broader impacts of the project stem from its potential to broaden adaptation of program analysis advances through the increased confidence of their evaluations; to lower the entry barrier for new program analysis researchers; and to accelerate the pace in program analysis research. PAClab's educational benefits include the ability for students to easily locate real-world programs and evaluate their implementations of program analyzers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.