Collaborative Research: Elements: Towards A Scalable Infrastructure for Archival and Reproducible Scientific Visualizations

今天的科学围绕着前沿数据集展开——科学家需要仔细分析这些数据，以便得出可靠的科学结论。这些前沿数据集变得越来越大、越来越复杂的速度每天都在加快。在许多方面，访问数据集并不等于，甚至接近于访问数据集中的见解。这种细微但至关重要的可获得性差异为科学结果的可重复性创造了深刻的障碍。为此，2010年发表在《科学》杂志上的“无障碍可重复性研究”提出了一个可重复性研究的系统。十年后，不幸的是，可获得的可重复性研究仍处于起步阶段。事实证明，这个障碍比以前认为的要根本得多，尽管从表面上看，它似乎可以通过投入资源和制定指导方针和政策来解决。真正的挑战在于，原始科学家团队的计算工具集、工作环境和工作流程，对于不同的科学家团队来说，很难精确地重现。这种困难源于计算机技术的快速发展；因此，以实际的方式冻结计算环境几乎是不可能的。此外，科学直觉很难编纂，简单地记录一个新想法不足以传达科学家在追求这个想法之前所看到的东西。从这个角度来看，要使可重复的研究成为现实，需要更好的方法和工具。在这个项目中，研究人员将专注于数据分析的可视化步骤，这是科学发现的核心组成部分。这个项目的目标是开发一个可复制交互式可视化（AIRIV）的归档基础设施。通过这种基础设施，研究人员将演示如何可靠地捕获、有效地存储、轻松地共享和自由地由任何用户重用大型和复杂数据的可视化探索。该项目将提高可重复性研究的可及性，促进科学进步。对于医学和药物研究等领域，该项目将为加速转化研究和促进国民健康提供前所未有的渠道。该项目将建立在先前NSF CISE研究基础设施奖资助的研究基础上。在之前的项目中，研究人员发现了一种方法来捕获可视化工具的交互式用户体验，并在不需要共享原始软件或原始数据的情况下共享捕获的体验。此外，在重用捕获的体验期间，用户可以自由地探索前一个用户如何使用称为Loom的方法使用该工具的确切顺序。在这个创建AIRIV的新项目中，研究人员将专注于基于网络的可视化仪表板，它代表了世界各地科学家与数据交互并获得见解的标准方式。这个项目将首先构建一个通用的AIRIV Javascript库，它可以被任何基于web浏览器的应用程序导入。使用AIRIV库，基于web的可视化仪表板的开发人员可以轻松地将Loom对象自动生成到他们的仪表板中。开发人员将能够为他们的应用程序提供工具，以便使用Loom对象存储新的来源信息。然后，研究人员将进行性能和扩展测试，以了解在本地、机构集群和社区共享数据基础设施设置下的托管选择之间的权衡。科学设施的经营者可以利用这些发现来帮助科学界做出明智的选择，以便在哪里以及如何托管科学可视化档案，从而提高可共享性和成本效率。研究人员还将开发机器学习方法，可以比较织机对象，并在整个织机对象档案中外部化共性和模式。这样的新方法将为AIRIV档案创建一个按示例搜索的功能。对于需求收集、持续改进和部署测试，研究人员将与Mayo Clinic & Illinois Alliance合作，该联盟是医疗保健领域几种技术的框架，其中许多技术都围绕着仪表板/分析工具的研究和开发。我们的目标是两个这样的分析工作，OmiX和KnowEnG，它们都是由国家超级计算应用中心（NCSA）开发的。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Today’s science revolves around leading edge datasets – data that scientists need to carefully analyze so that they can draw reliable scientific conclusions. The rate at which these leading-edge datasets are becoming larger and more complex is accelerating every day. In many ways, having access to a dataset does not equal to, or even come close to, having access to the insights in the dataset. This nuanced but crucial difference in accessibility creates a deep barrier to making scientific results reproducible. To this end, “Accessible Reproducible Research”, published by Science in 2010, presented a system for reproducible research. A decade later, unfortunately, accessible reproducible research is still in its infancy. It turns out that this barrier is much more fundamental than previously believed, even though on the surface it seems solvable by investing resources and setting guidelines and policies. The real challenge is that the computing toolsets, the working environments, and the work processes of the original team of scientists are very difficult for a different team of scientists to recreate with precision. Such difficulty stems from the rapid speed at which computing technology is advancing; so that freezing a computing environment in a practical manner is nearly impossible. In addition, scientific intuition is difficult to codify, simply documenting a new idea is not enough to communicate what a scientist saw before pursuing that idea. From that respect, making accessible reproducible research a reality requires better methods and tools. In this project, the investigators will focus on the visualization step of data analysis, which is a central component of scientific discovery. This project’s aim is to develop an Archiving Infrastructure for Reproducible Interactive Visualization (AIRIV). Through this infrastructure, the investigators will demonstrate how visual explorations of large and complex data can be reliably captured, efficiently stored, easily shared, and freely reused by any user. This project will improve accessibility of reproducible research and promote the progress of science. For areas such as medicine and pharmaceutical research, this project will provide an unprecedented channel to accelerate translational research and advance the national health.This project will build upon research funded by a prior NSF CISE Research Infrastructure award. In that previous project, the investigators found a method to capture interactive user experience of visualization tools, and to share the captured experience without the need to share the original software or the original data. Furthermore, during the reuse of a captured experience, the user has freedom to explore beyond the exact sequence of how the previous user has used the tool with a method called Loom. In this new project to create AIRIV, the investigators will focus on web-based visualization dashboards, which represent the standard way for scientists around the world to interact with their data and derive insights. This project will first build a general AIRIV Javascript library that can be imported by any web browser-based application. Using the AIRIV library, developers of web-based visual dashboards can easily implement automatic generation of Loom objects into their dashboards. Developers will be able to instrument their applications to store new provenance information with Loom objects as well. The investigators will then conduct performance and scaling tests to understand the tradeoffs between hosting choices under settings of local, institutional clusters, and community shared data infrastructures. Operators of scientific facilities can use the findings to help science communities make informed choices as to where and how to host scientific visualization archives for better share-ability and cost efficiency. The investigators will also develop machine learning methods that can compare Loom objects and externalize commonalities and patterns in an entire archive of Loom objects. Such new methods will lead to creating a search by example functionality for AIRIV archives. For requirements collection, continuous improvement, and deployment testing, the investigators will engage the Mayo Clinic & Illinois Alliance, which serves as a framework for several technologies in healthcare, many of which center around the research and development of dashboard/analytical tools. We target two such analytics efforts, OmiX and KnowEnG, both of which are developed at National Center for Supercomputing Applications (NCSA).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.