EAGER: Improving Reproducibility of Computing Research using Proxy Workloads

计算研究很少被复制、揭穿或解构。该项目探讨了使用小型化代理工作负载来提高计算研究的可重复性。过去的研究表明，微型代理工作负载在前硅设计探索中是有用的。通过创建改进的代理和传播代理和代理生成器，该项目旨在从再现性角度探索代理的使用。代理可以使再现的研究与减少的努力，进行研究与广泛的基准，提高耐用性/鲁棒性的研究，通过覆盖大的工作量空间，并通过提供更多的机会，模拟器校准和验证。具有细粒度代理、覆盖广泛工作负载空间的代理以及验证特定架构方面的定向代理的存储库可能会导致采用代理，这将在可再现性方面迈出一大步。拟议的探索可能会改变计算研究的进行和验证方式。 目前有最小的努力，计算研究的可重复性，虽然ACM最近制定了一个可重复性的倡议。拟议的项目可以通过使用代理工作负载来提高计算研究的可重复性。 社区中的研究人员将能够更好地验证他们自己的工作，并在没有大量时间/精力的情况下复制他人的工作。可再现性的其他障碍，例如缺乏对专有基准的可访问性，在工作负载空间的许多区域中缺乏基准等，可以使用小型化的代理工作负载来解决。 在该项目中创建的知识库将使计算研究人员能够提高其研究结果的可重复性，鲁棒性，可靠性和普遍性。沿着研究活动，将设计一个教育和传播方案，向学生、研究人员以及更广大的受众宣传这项工作的成果，并将代表性不足社区的学生包括在内。研究结果将在PI的网站上发布。

Computing research is very minimally reproduced, debunked, or deconstructed. This project explores the use of miniaturized proxy workloads in improving the reproducibility of computing research. Past research has shown the usefulness of miniaturized proxy workloads in pre-silicon design explorations. By creating improved proxies and by disseminating the proxies and the proxy generator, this project aims to explore usage of proxies from the reproducibility angle. The proxies can enable reproducing studies with reduced effort, conducting studies with a broad range of benchmarks, improving ruggedness/robustness of the study by covering a large workload space, and by offering more opportunities for simulator calibration and validation. A repository with fine-grain proxies, proxies that cover a broad workload space, and directed proxies that validate specific architectural aspects, is likely to lead to adoption of proxies which will make a big stride towards reproducibility. The proposed explorations will likely transform how computing research is conducted and validated. Currently there is minimal effort on reproducibility of computing research, although ACM has recently instituted a reproducibility initiative. The proposed project can make a broad impact in improving the reproducibility of computing research by the use of proxy workloads. Researchers in the community will be enabled to better validate their own work, and reproduce others' work without an overwhelming amount of time/effort. Other barriers towards reproducibility such as lack of accessibility to proprietary benchmarks, lack of benchmarks in many regions of the workload space, etc. can be addressed using miniaturized proxy workloads. The repository created in the project will enable computing researchers to enhance repeatability, robustness, reliability and generalizability of their research findings. Along with the research activities, an educational and dissemination program will be designed to communicate the results of this work to students, researchers, as well as a more general audience, and to include students from underrepresented communities. The research results will be published at the PI's website.