EAGER: Improving Reproducibility of Computing Research using Proxy Workloads

计算研究被极少地复制、揭穿或解构。这个项目探索了使用小型化的代理工作负载来提高计算研究的重现性。过去的研究已经表明，小型化的代理工作负载在预硅设计探索中是有用的。通过创建改进的代理并传播代理和代理生成器，该项目旨在从可重现性的角度探索代理的使用。这些代理能够以更少的工作量重现研究，使用广泛的基准进行研究，通过覆盖较大的工作负载空间来提高研究的坚固性/稳健性，并通过提供更多的模拟器校准和验证机会。具有细粒度代理、覆盖广泛工作负载空间的代理以及验证特定体系结构方面的定向代理的存储库可能会导致采用代理，从而向可重现性迈进一大步。拟议中的探索可能会改变计算研究的进行和验证方式。目前，尽管ACM最近发起了一项可再生性倡议，但在计算研究的再现性方面所做的努力很少。拟议的项目可以通过使用代理工作负载在提高计算研究的重现性方面产生广泛的影响。社区中的研究人员将能够更好地验证自己的工作，并在不花费大量时间/精力的情况下复制他人的工作。可重复性方面的其他障碍，例如无法访问专有基准、工作负载空间的许多区域缺乏基准等，都可以使用小型代理工作负载来解决。在该项目中创建的存储库将使计算研究人员能够提高其研究成果的重复性、稳健性、可靠性和普适性。除了研究活动，还将设计一项教育和传播计划，向学生、研究人员以及更广泛的受众传达这项工作的结果，并包括来自代表性不足社区的学生。研究结果将在国际刑警组织的网站上公布。

Computing research is very minimally reproduced, debunked, or deconstructed. This project explores the use of miniaturized proxy workloads in improving the reproducibility of computing research. Past research has shown the usefulness of miniaturized proxy workloads in pre-silicon design explorations. By creating improved proxies and by disseminating the proxies and the proxy generator, this project aims to explore usage of proxies from the reproducibility angle. The proxies can enable reproducing studies with reduced effort, conducting studies with a broad range of benchmarks, improving ruggedness/robustness of the study by covering a large workload space, and by offering more opportunities for simulator calibration and validation. A repository with fine-grain proxies, proxies that cover a broad workload space, and directed proxies that validate specific architectural aspects, is likely to lead to adoption of proxies which will make a big stride towards reproducibility. The proposed explorations will likely transform how computing research is conducted and validated. Currently there is minimal effort on reproducibility of computing research, although ACM has recently instituted a reproducibility initiative. The proposed project can make a broad impact in improving the reproducibility of computing research by the use of proxy workloads. Researchers in the community will be enabled to better validate their own work, and reproduce others' work without an overwhelming amount of time/effort. Other barriers towards reproducibility such as lack of accessibility to proprietary benchmarks, lack of benchmarks in many regions of the workload space, etc. can be addressed using miniaturized proxy workloads. The repository created in the project will enable computing researchers to enhance repeatability, robustness, reliability and generalizability of their research findings. Along with the research activities, an educational and dissemination program will be designed to communicate the results of this work to students, researchers, as well as a more general audience, and to include students from underrepresented communities. The research results will be published at the PI's website.