Collaborative Research: CCRI: New: World Of Code (WoC): The development of curated code resource to support research in software engineering

该项目将创建公共研究基础设施，围绕实现源代码和版本控制历史数据的精心收集，近似于整个开源软件（OSS）。在过去的二十年里，来自开源软件开发的真实数据促进了软件工程研究的进步。尽管OSS版本控制数据是公开和详细的（包括开发人员的操作和源代码的版本），但庞大的规模和对策展（收集，上下文化，更正，增强和集成）的需求使得这些数据不适合研究。数据分布在许多平台上，嵌入在许多工具和格式中，并分布在数千万个存储库中。此外，在整个OSS生态系统中管理数据的难度超出了单个研究小组的能力，也留下了许多重要的研究问题没有答案。各个OSS项目相互依赖，共享源代码和开发人员。这造成了巨大的风险，例如易受攻击的源代码的传播和志愿维护者脱离的涟漪反应。该团队将创建几乎完整、全面策划和广泛交叉引用的版本控制数据，使研究社区能够衡量和理解OSS生态系统的动态，从而帮助识别和管理特别是OSS和整个社会的风险。该项目将使用软件工程社区的投入创建一个研究基础设施，其中包括：1）定期更新和交叉引用的源代码和版本历史资源，接近整个OSS; 2）数据管理能力，例如，身份消除歧义和依赖关系提取; 3）易于使用的Web服务和应用程序，以支持常见的研究任务; 4）培训：教程，指导，黑客马拉松和研讨会，以帮助有效和高效地使用资源; 5）研究人员，开发人员和公司的社区，他们维护，指导，增强和运营这个基础设施。这将有助于回答一系列全新的研究问题，这些问题涉及由技术依赖性、代码共享和知识流定义的OSS网络结构。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will create public research infrastructure centered around achieving a curated collection of source-code and version control history data approximating the entirety of open-source software (OSS). Real-world data from open-source software development has catalyzed progress in software engineering research in the last two decades. Despite the OSS version control data being public and detailed (with actions of developers and versions of the source code), the sheer scale and the need for curation (collection, contextualization, correction, augmentation, and integration) make such data unsuitable for research. The data are spread across many platforms, embedded in many tools and formats, and spread across tens of millions of repositories. Moreover, the difficulty of curating data across the entire OSS ecosystem, beyond the capabilities of individual research groups, also leaves many important research questions unanswered. Individual OSS projects depend on each other and share source code and developers among them. This creates tremendous risks, for example the spread of vulnerable source code and the ripple effects of volunteer maintainers disengaging. The team will create nearly complete, fully curated, and extensively cross-referenced version control data that will enable the research community to measure and understand the dynamics of OSS ecosystems and, thus, help identify and manage risk to OSS in particular and to society in general.This project will use input from the software engineering community to create a research infrastructure that contains: 1) regularly updated and cross-referenced source-code and version history resource approximating the entirety of OSS; 2) data curation capabilities, e.g., identity disambiguation and extraction of dependencies; 3) easy-to-use web services and applications to support common research tasks; 4) training: tutorials, mentoring, hackathons and seminars to help use the resource effectively and efficiently; 5) a community of researchers, developers, and companies who maintain, guide, enhance, and operate this infrastructure. This will enable answers to an entirely new set of research questions concerning OSS network structure defined by technical dependencies, code sharing, and knowledge flows. It will also provide accessible means for stratified sampling from the OSS universe of code, improving the generality of research findings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.