EAGER: Improving Reproducibility of Computing Research using Proxy Workloads

计算机研究很少被复制、揭露或解构。这个项目探索了小型化代理工作负载的使用，以提高计算研究的可重复性。过去的研究表明，小型化代理工作负载在预硅设计探索中的有用性。通过创建改进的代理，并通过传播代理和代理生成器，该项目旨在从可再现性的角度探索代理的使用。代理可以减少重复研究的工作量，在广泛的基准范围内进行研究，通过覆盖更大的工作负载空间来提高研究的坚固性/稳健性，并提供更多的模拟器校准和验证机会。具有细粒度代理、覆盖广泛工作负载空间的代理和验证特定体系结构方面的定向代理的存储库很可能导致采用代理，从而在可再现性方面迈出一大步。提出的探索可能会改变计算研究的进行和验证方式。目前在计算研究的可再现性方面的努力很少，尽管ACM最近建立了一个可再现性倡议。提议的项目可以通过使用代理工作负载在提高计算研究的可重复性方面产生广泛的影响。社区中的研究人员将能够更好地验证他们自己的工作，并在没有大量时间/精力的情况下复制他人的工作。其他妨碍再现性的障碍，如专有基准缺乏可访问性、在工作负载空间的许多区域缺乏基准等，都可以使用小型化的代理工作负载来解决。该项目中创建的存储库将使计算研究人员能够提高其研究成果的可重复性、健壮性、可靠性和通用性。在开展研究活动的同时，还将制定一项教育和传播计划，向学生、研究人员以及更广泛的受众，包括来自代表性不足社区的学生，传达这项工作的成果。研究结果将在PI的网站上公布。

Computing research is very minimally reproduced, debunked, or deconstructed. This project explores the use of miniaturized proxy workloads in improving the reproducibility of computing research. Past research has shown the usefulness of miniaturized proxy workloads in pre-silicon design explorations. By creating improved proxies and by disseminating the proxies and the proxy generator, this project aims to explore usage of proxies from the reproducibility angle. The proxies can enable reproducing studies with reduced effort, conducting studies with a broad range of benchmarks, improving ruggedness/robustness of the study by covering a large workload space, and by offering more opportunities for simulator calibration and validation. A repository with fine-grain proxies, proxies that cover a broad workload space, and directed proxies that validate specific architectural aspects, is likely to lead to adoption of proxies which will make a big stride towards reproducibility. The proposed explorations will likely transform how computing research is conducted and validated. Currently there is minimal effort on reproducibility of computing research, although ACM has recently instituted a reproducibility initiative. The proposed project can make a broad impact in improving the reproducibility of computing research by the use of proxy workloads. Researchers in the community will be enabled to better validate their own work, and reproduce others' work without an overwhelming amount of time/effort. Other barriers towards reproducibility such as lack of accessibility to proprietary benchmarks, lack of benchmarks in many regions of the workload space, etc. can be addressed using miniaturized proxy workloads. The repository created in the project will enable computing researchers to enhance repeatability, robustness, reliability and generalizability of their research findings. Along with the research activities, an educational and dissemination program will be designed to communicate the results of this work to students, researchers, as well as a more general audience, and to include students from underrepresented communities. The research results will be published at the PI's website.