SI2-SSI: Integrating the NIMBLE Statistical Algorithm Platform with Advanced Computational Tools and Analysis Workflows

该项目开发的软件将使科学家能够从复杂的数据中了解更多信息，并更容易地分享新的分析方法。 越来越多的科学家在许多领域的目标是从大型和复杂的数据集得出合理的结论。 这些领域包括环境生物学、政治学、教育研究、大气和海洋科学、气候科学等。 数据可能是复杂的，因为许多相关的变量被测量和/或因为一些测量不是彼此独立的。 非独立性可能出现在以下情况：一些变量在一段时间内被重复测量;或在附近地点进行测量;或对相关个体群体进行测量;或出于这些原因和其他类似原因的组合。 对于这种情况，一般的统计方法已经开发出来，使研究人员能够调整他们的分析，以每个数据集，以说明数据之间的关系。 鉴于有限数据中固有的不确定性，这种方法依赖于计算机算法来探索可能的结论范围。在这些一般方法中，有许多种类的具体办法已经并将继续得到发展。 因此，出现了一个主要的软件差距：许多新的和不断发展的方法不容易被广泛的科学家应用，因为没有一个软件框架，使它们易于编程和传播。 该项目将支持继续开发NIMBLE软件，以帮助填补这一空白。 因此，科学家将能够更灵活地使用计算分析方法，更容易地联合收割机和比较不同的算法，将此类算法集成到其他软件工作流程中，并获得更好的计算性能。 这将使更先进和更常规的使用一些现代计算方法来分析复杂的data.The现有的NIMBLE框架分层统计模型和算法包括一个模型规范语言，一个语言的编程模型通用算法在R统计环境，和编译器，生成，编译和接口到模型和算法特定的C++高效执行。 这使得一般的实施和传播的方法，如马尔可夫链蒙特卡罗，顺序蒙特卡罗，和许多相关的方法。在这个项目中，NIMBLE将被扩展和推广，变得更加强大和灵活，从而能够在各种软件工作流程中使用。 对NIMBLE核心能力的扩展将包括利用生成的C++中的自动微分和并行化，增强其现有的线性代数能力，更有效地实现大型统计模型，包括具有结构不确定性（如潜在组成员资格）的模型，以及对统计建模语言的扩展。 促进NIMBLE生成的模型和算法与其他软件集成的增强功能将包括生成独立的可执行文件，生成明确定义的应用程序接口（例如供Python使用），从算法代码调用用户提供的库的功能，通过JSON和NetCDF等标准格式加载和保存数据的功能，以及将NIMBLE组件分离到不同的包中。该项目将包括大量的外联、培训和用户社区发展。 这些活动将包括人口和生态系统生态学、海洋学、气候科学、政治科学和教育等领域的用例开发。这些活动还将包括讲习班、用户会议、关键用户访问和培训材料。该奖项由高级网络基础设施部门联合支持，由NSF数学和物理科学理事会（数学科学部）。

The software developed in this project will enable scientists to learn more from complex data and to share new analysis methods more easily. Increasingly, scientists in many fields aim to draw sound conclusions from large and complex data sets. Such fields include environmental biology, political science, education research, atmospheric and oceanic science, climate science, and many others. Data may be complex because many related variables are measured and/or because some measurements are not independent from each other. Non-independence can arise when some variables are measured repeatedly through time; or when measurements are made at nearby locations; or when measurements are made on groups of related individuals; or for a combination of those and other similar reasons. For such cases, general statistical methods have been developed to allow researchers to tailor their analysis to each data set in order to account for the relationships among the data. Such methods rely on computer algorithms to explore the range of possible conclusions given the uncertainties inherent in limited data. Within those general methods there are many varieties of specific approaches that have been and continue to be developed. Thus, a major software gap has emerged: Many new and evolving methods are not easily available for application by a wide range of scientists because there has not been a software framework that makes them easy to program and disseminate. This project will support continued development of the NIMBLE software to help fill that gap. As a result, scientists will be able to use computational analysis methods more flexibly, to combine and compare different algorithms more easily, to integrate such algorithms into other software workflows, and to gain better computational performance. This will enable more advanced and more routine use of some modern computational methods for analyzing complex data.The existing NIMBLE framework for hierarchical statistical models and algorithms comprises a model specification language, a language for programming model-generic algorithms within the R statistical environment, and a compiler that generates, compiles and interfaces to model- and algorithm-specific C++ for efficient execution. These enable general implementation and dissemination of methods such as Markov chain Monte Carlo, sequential Monte Carlo, and many related methods. In this project NIMBLE will be extended and generalized to be more powerful and flexible, enabling use in a variety of software workflows. Extensions to NIMBLE's core capabilities will include harnessing automatic differentiation and parallelization in generated C++, enhancements to its existing linear algebra capabilities, more efficient implementation of large statistical models including those with structural uncertainty such as latent group membership, and extensions to the statistical modeling language. Enhancements to facilitate integration of NIMBLE-generated models and algorithms with other software will include generation of stand-alone executables, generation of clearly defined application-programmer interfaces such as for use by Python, features to call user-provided libraries from algorithm code, features to load and save data via standard formats such as JSON and NetCDF, and separation of NIMBLE components into distinct packages. The project will include substantial outreach, training, and user community development. These activities will include development of uses cases in fields such as population and ecosystem ecology, oceanography, climate science, political science, and education. They will also include workshops, user meetings, key-user visits, and training material. This award by the Advanced Cyberinfrastructure Division is jointly supported by the NSF Directorate for Mathematical and Physical Sciences (Division of Mathematical Sciences).