SI2-SSI: Integrating the NIMBLE Statistical Algorithm Platform with Advanced Computational Tools and Analysis Workflows

在这个项目中开发的软件将使科学家能够从复杂的数据中了解更多，并更容易地分享新的分析方法。越来越多的科学家在许多领域致力于从庞大而复杂的数据集中得出可靠的结论。这些领域包括环境生物学、政治学、教育研究、大气和海洋科学、气候科学以及许多其他领域。数据可能是复杂的，因为测量了许多相关的变量和/或因为一些测量不是相互独立的。当一些变量在一段时间内反复测量时，就会产生非独立性；或者在附近地点进行测量时；或者对相关个体群体进行测量；或者是这些和其他类似原因的结合。对于这种情况，一般的统计方法已经开发出来，使研究人员能够根据每个数据集定制他们的分析，以便解释数据之间的关系。这种方法依靠计算机算法来探索有限数据中固有的不确定性所带来的可能结论的范围。在这些一般方法中，有许多不同种类的具体方法已经并将继续发展。因此，一个主要的软件缺口出现了：许多新的和不断发展的方法不容易被广泛的科学家应用，因为没有一个软件框架使它们易于编程和传播。该项目将支持敏捷软件的持续开发，以帮助填补这一空白。因此，科学家将能够更灵活地使用计算分析方法，更容易地组合和比较不同的算法，将这些算法集成到其他软件工作流中，并获得更好的计算性能。这将使一些现代计算方法能够更高级、更常规地用于分析复杂数据。现有的用于分层统计模型和算法的敏捷框架包括一种模型规范语言，一种在R统计环境中用于编程模型通用算法的语言，以及一个编译器，该编译器生成、编译和接口到特定于模型和算法的c++，以实现高效执行。这使得一般实现和传播的方法，如马尔可夫链蒙特卡罗，顺序蒙特卡罗，和许多相关的方法。在这个项目中，敏捷将被扩展和推广为更强大和灵活，能够在各种软件工作流程中使用。对NIMBLE核心功能的扩展将包括在生成的c++中利用自动分化和并行化，增强其现有的线性代数功能，更有效地实现大型统计模型，包括那些具有结构不确定性（如潜在组成员）的模型，以及对统计建模语言的扩展。促进NIMBLE生成的模型和算法与其他软件集成的增强功能将包括生成独立可执行文件、生成明确定义的应用程序程序员接口（如Python）、从算法代码调用用户提供的库的功能、通过标准格式（如JSON和NetCDF）加载和保存数据的功能，以及将NIMBLE组件分离到不同的包中。该项目将包括大量的外联、培训和用户社区发展。这些活动将包括在人口和生态系统生态学、海洋学、气候科学、政治学和教育等领域开发用例。它们还将包括讲习班、用户会议、关键用户访问和培训材料。该奖项由高级网络基础设施部颁发，由美国国家科学基金会数学和物理科学理事会（数学科学部）联合支持。

The software developed in this project will enable scientists to learn more from complex data and to share new analysis methods more easily. Increasingly, scientists in many fields aim to draw sound conclusions from large and complex data sets. Such fields include environmental biology, political science, education research, atmospheric and oceanic science, climate science, and many others. Data may be complex because many related variables are measured and/or because some measurements are not independent from each other. Non-independence can arise when some variables are measured repeatedly through time; or when measurements are made at nearby locations; or when measurements are made on groups of related individuals; or for a combination of those and other similar reasons. For such cases, general statistical methods have been developed to allow researchers to tailor their analysis to each data set in order to account for the relationships among the data. Such methods rely on computer algorithms to explore the range of possible conclusions given the uncertainties inherent in limited data. Within those general methods there are many varieties of specific approaches that have been and continue to be developed. Thus, a major software gap has emerged: Many new and evolving methods are not easily available for application by a wide range of scientists because there has not been a software framework that makes them easy to program and disseminate. This project will support continued development of the NIMBLE software to help fill that gap. As a result, scientists will be able to use computational analysis methods more flexibly, to combine and compare different algorithms more easily, to integrate such algorithms into other software workflows, and to gain better computational performance. This will enable more advanced and more routine use of some modern computational methods for analyzing complex data.The existing NIMBLE framework for hierarchical statistical models and algorithms comprises a model specification language, a language for programming model-generic algorithms within the R statistical environment, and a compiler that generates, compiles and interfaces to model- and algorithm-specific C++ for efficient execution. These enable general implementation and dissemination of methods such as Markov chain Monte Carlo, sequential Monte Carlo, and many related methods. In this project NIMBLE will be extended and generalized to be more powerful and flexible, enabling use in a variety of software workflows. Extensions to NIMBLE's core capabilities will include harnessing automatic differentiation and parallelization in generated C++, enhancements to its existing linear algebra capabilities, more efficient implementation of large statistical models including those with structural uncertainty such as latent group membership, and extensions to the statistical modeling language. Enhancements to facilitate integration of NIMBLE-generated models and algorithms with other software will include generation of stand-alone executables, generation of clearly defined application-programmer interfaces such as for use by Python, features to call user-provided libraries from algorithm code, features to load and save data via standard formats such as JSON and NetCDF, and separation of NIMBLE components into distinct packages. The project will include substantial outreach, training, and user community development. These activities will include development of uses cases in fields such as population and ecosystem ecology, oceanography, climate science, political science, and education. They will also include workshops, user meetings, key-user visits, and training material. This award by the Advanced Cyberinfrastructure Division is jointly supported by the NSF Directorate for Mathematical and Physical Sciences (Division of Mathematical Sciences).