Elements: Software: Distributed Workflows for Cyberexperimentation

计算系统、网络和安全方面的工业和学术研究无法证明和验证研究成果，因为研究过程包括复杂但主要是在分布式、大规模、网络化的试验台上进行的实验。 该项目将通过将实验定义为可执行的工作流程，从而在测试台上进行严格和可重复的实验来解决这个问题，然后可以重复运行和管理。该项目将开发和部署一个名为Elie的软件框架。Elie包含一个实验表示功能，沿着几个支持服务，使实验工作流程健壮，容错，易于共享和可重用。 具体来说，Elie包括：（1）一个新的实验表示，称为DEW，它编码的实验行为和分配约束，而不仅仅是实验拓扑。(2)捕获工具，用于捕获研究人员在试验台上的手动操作并促进脚本构建，以及DEW生成（3）试验台集成版本控制，（4）实验编排机制，（5）轻量级故障检测，（6）GUI，允许在整个实验生命周期中进行实验管理。Elie的技术可以被研究人员用作完整的环境或个人工具。 这些技术共同在整个实验生命周期（包括设计、运行、分析、监控、存储和共享阶段）中提供基本的测试平台支持。 Elie的所有技术都是可扩展的，并且是开源的，允许研究社区为他们的开发做出贡献，并根据他们的需求进行定制。 该项目从根本上不同于先前的工作，其目标是与当前的测试台实验方法（手动和脚本实验）共存，并适用于广泛的测试台（通过其翻译和生成工具）。该项目的成果将通过将繁琐，重复和细节敏感的任务卸载到测试台上，显着提高测试台实验的便利性。 这将缩短实验时间，提高结果的质量和可靠性。 这项工作将进一步使实验更加强大，使用户能够以最小的努力轻松共享和重复使用实验，并促进重复性和再现性。总的来说，试验台实验的这种进步将使分布式系统、网络和网络安全等使用试验台的领域能够进行垂直开发（以他人的工作为基础）。共享和再利用还将提高这些领域的科学研究质量，因为可以在其他研究人员共享的人工制品的基础上创建更大、更复杂的实验。该项目团队还将开展广泛的用户推广活动，并与致力于提高试验台实验严谨性的其他研究人员建立合作关系，目标是为Elie建立一个强大的用户和开发人员社区。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Industry and academic research in computing systems, networking and security suffers from the inability to demonstrate and validate research outcomes, because the process of research consists of complex but largely experiments on distributed, large-scale, networked test-beds. This project will address this issue by enabling rigorous and repeatable experimentation on test-beds by enabling experiments to be defined as executable workflows, that can then be repeatably run, as well as managed. This project will develop and deploy a software framework called Elie. Elie consists of an experiment representation capability, along with several supporting services that make experimentation workflows robust, fault tolerant, easily sharable and reusable. Specifically, Elie consists of: (1) A new experiment representation, called DEW, which encodes experiment behavior and allocation constraints instead of just experiment topology. (2) Capture tools, which capture a researcher's manual actions on the testbed and facilitate script-building, and DEW generation (3) Testbed-integrated version control, (4) An experiment orchestration mechanism, (5) Lightweight failure detection, (6) A GUI, which allows for experiment management throughout an experiment lifecycle. Elie's technologies can be used by researchers as a full environment or as individual tools. Collectively these technologies provide basic testbed support throughout an experiment lifecycle including the design, running, analysis, monitoring, storage and sharing stages. All of Elie's technologies are extensible, and open-sourced, to allow the research community to contribute to their development and customize them to their needs. This project fundamentally differs from the prior work in its goal to co-exist with current testbed experimentation approaches (manual and scripted experimentation) and to be applicable to a broad range of testbeds (via its translation and generation tools). Outcomes of this project will significantly improve ease of testbed experimentation, by offloading tedious, repetitive and detail-sensitive tasks to testbeds. This will shorten experiment duration and improve quality and reliability of results. This work will further make experiments more robust, enable easy sharing and reuse of experiments with minimal user effort, and facilitate repeatability and reproducibility. Overall, such advances in testbed experimentation will enable vertical development (building upon the work of others) in fields, which use testbeds, such as distributed systems, networking and cybersecurity. Sharing and reuse will also improve quality of scientific research in these fields, by enabling creation of larger and more complex experiments built on the shared artifacts of other researchers. The project team will also conduct extensive user outreach and building collaborations with other researchers that work on improving rigor of testbed experimentation, with the goal of building a strong user and developer community for Elie.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.