POSE: Phase II: Expanding the data.table ecosystem for efficient big data manipulation in R

北亚利桑那大学的Toby Hocking教授获得了技术、创新和伙伴关系理事会（TIP）的开源生态系统（POSE）项目的奖励。系统地收集和记录的数据比人类历史上任何时候都要多，需要使用有限的计算资源来存储和分析这些数据的高效软件包。一个领先的、最先进的例子是数据。table，这是一个免费/开源的软件，用于内存中的数据操作/分析，用C代码作为R包实现，在计算时间和内存使用方面都非常高效。尽管数据。table拥有大量的用户（包括数千个从数据中导入功能的其他R包）。表)，数据的增长情况。Table受到以下因素的限制：(1)其扁平/非正式的领导结构，只有一个作者在顶层，他可以批准新的代码贡献；(2)缺乏文档/翻译和促进多样性/包容性的社区标准；(3)缺乏系统软件测试的基础设施。该项目将扩展数据用户、贡献者和开发者的开源生态系统。表，通过解决这些问题。特别地，该项目将创建(1)具有新的分层领导结构的书面治理文档，(2)用于新用户/贡献者的新文档材料，包括翻译和鼓励多样性/包容性的社区标准，以及(3)新的测试软件和基础设施。此外，该项目还包括系统地评估这项工作对数据的广泛影响的计划。通过衡量重要指标（独特贡献者的数量、贡献者的多样性、依赖包的数量等）的变化来表生态系统。这个项目的结果将是一个自我维持的开源数据生态系统。表，这将使它成长为一个更强大的数据分析工具，将被更多的人使用，更多样化的人，在未来。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Professor Toby Hocking from Northern Arizona University is supported by an award from the Pathways to Enable Open-Source Ecosystems (POSE) program in the Directorate for Technology, Innovation and Partnerships (TIP). More data is being systematically gathered and recorded than at any previous time in human history, and efficient software packages are required to store and analyze these data using limited computational resources. A leading, state-of-the-art example is data.table, which is free/open-source software for in-memory data manipulation/analysis, implemented as an R package with C code that is highly efficient in terms of both computation time and memory usage. Although data.table has a substantial number of users (including thousands of other R packages which import functionality from data.table), the growth of data.table is limited by (1) its flat/informal leadership structure with only one author at the top who can approve new code contributions, (2) lack of documentation/translations and community standards for promoting diversity/inclusion, and (3) lack of infrastructure for systematic software testing. This project will expand the open-source ecosystem of users, contributors, and developers of data.table, by addressing these issues. In particular, this project will create (1) a written governance document with a new hierarchical leadership structure, (2) new documentation materials for onboarding new users/contributors, including translations and community standards to encourage diversity/inclusion, and (3) new testing software and infrastructure. Furthermore, the project includes plans to systematically evaluate the broader impacts of this effort on the data.table ecosystem by measuring changes to important metrics (number of unique contributors, diversity of contributors, number of dependent packages, etc). The result of this project will be a self-sustaining open-source ecosystem for data.table, which will allow it to grow into a more powerful data analysis tool that will be used by more people, and more diverse people, in the future.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.