Collaborative Research: CCRI: New: World Of Code (WoC): The development of curated code resource to support research in software engineering

该项目将创建公共研究基础设施，以实现接近整个开放源码软件(OSS)的源代码和版本控制历史数据的精心收集。在过去的二十年里，来自开源软件开发的真实世界数据促进了软件工程研究的进步。尽管OSS版本控制数据是公开和详细的(包括开发人员的操作和源代码的版本)，但纯粹的规模和管理(收集、上下文、更正、增强和集成)的需要使得这些数据不适合研究。这些数据分布在许多平台上，嵌入在许多工具和格式中，分布在数千万个存储库中。此外，在整个开放源码软件生态系统中管理数据的难度超出了各个研究小组的能力范围，这也留下了许多重要的研究问题没有得到回答。各个OSS项目相互依赖，并在其中共享源代码和开发人员。这造成了巨大的风险，例如易受攻击的源代码的传播和志愿维护人员脱离的连锁反应。该团队将创建几乎完整的、经过充分管理的和广泛交叉引用的版本控制数据，这些数据将使研究团体能够测量和了解开放源码软件生态系统的动态，从而帮助识别和管理对开放源码软件特别和一般社会的风险。该项目将使用来自软件工程界的输入来创建研究基础设施，该基础设施包括：1)定期更新和交叉引用的源代码和版本历史资源，接近整个开放源码软件；2)数据管理能力，例如身份消除歧义和提取依赖关系；3)易于使用的网络服务和应用程序，以支持常见的研究任务；4)培训：教程、指导、黑客松和研讨会，以帮助有效和高效地使用资源；5)由维护、指导、增强和运营该基础设施的研究人员、开发人员和公司组成的社区。这将使人们能够回答一组关于开放源码软件网络结构的全新研究问题，这些网络结构由技术依赖、代码共享和知识流定义。它还将提供可访问的方法，从开放源码软件的代码宇宙中进行分层抽样，提高研究成果的通用性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project will create public research infrastructure centered around achieving a curated collection of source-code and version control history data approximating the entirety of open-source software (OSS). Real-world data from open-source software development has catalyzed progress in software engineering research in the last two decades. Despite the OSS version control data being public and detailed (with actions of developers and versions of the source code), the sheer scale and the need for curation (collection, contextualization, correction, augmentation, and integration) make such data unsuitable for research. The data are spread across many platforms, embedded in many tools and formats, and spread across tens of millions of repositories. Moreover, the difficulty of curating data across the entire OSS ecosystem, beyond the capabilities of individual research groups, also leaves many important research questions unanswered. Individual OSS projects depend on each other and share source code and developers among them. This creates tremendous risks, for example the spread of vulnerable source code and the ripple effects of volunteer maintainers disengaging. The team will create nearly complete, fully curated, and extensively cross-referenced version control data that will enable the research community to measure and understand the dynamics of OSS ecosystems and, thus, help identify and manage risk to OSS in particular and to society in general.This project will use input from the software engineering community to create a research infrastructure that contains: 1) regularly updated and cross-referenced source-code and version history resource approximating the entirety of OSS; 2) data curation capabilities, e.g., identity disambiguation and extraction of dependencies; 3) easy-to-use web services and applications to support common research tasks; 4) training: tutorials, mentoring, hackathons and seminars to help use the resource effectively and efficiently; 5) a community of researchers, developers, and companies who maintain, guide, enhance, and operate this infrastructure. This will enable answers to an entirely new set of research questions concerning OSS network structure defined by technical dependencies, code sharing, and knowledge flows. It will also provide accessible means for stratified sampling from the OSS universe of code, improving the generality of research findings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.