III: Large: Collaborative Research: SciDB - An Array Oriented Data Management System for Massive Scale Scientific Data

III：大型：协作研究：SciDB - 用于大规模科学数据的面向数组的数据管理系统

• 批准号: 1111423
• 负责人: Stanley Zdonik
• 金额: $ 73.7万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1111423&HistoricalAwards=false
• 关键词: III; Large; Collaborative; Research; SciDB

This collaborative project brings together expertise of five research teams at Brown University (IIS-1111423), University of Washington (IIS-1110370), Massachusetts Institute of Technology (IIS-1111371), Portland State University (IIS-1110917) and University of Wisconsin-Madison (IIS-1111423). Scientific data management has traditionally been performed using the file system, at best using files structured according to a low-level data format. Higher-level data management infrastructure has been task-specific and not reusable in different domains, resulting in millions of dollars of duplicated implementation effort by scientists to manage their data. The goal of this project is the development of a scientific database (SciDB), a system designed and optimized for scientific applications. The aim of SciDB is to do for science what relational databases did for the business world, namely to provide a high performance, commercial-quality and scalable data management system appropriate for many science domains.In contrast to existing database systems, SciDB is based on a multidimensional array data model and includes multiple features specific to science and critical for science: provenance, uncertainty, versions, time travel, science-specific operations, and in situ data processing. No existing system offers all these features in a single, highly scalable engine. SciDB thus significantly advances the state-of-the-art in data management in addition to supporting domain scientists in data-driven knowledge discovery. The intellectual merit of SciDB is in exploring novel, high performance solutions to nested array storage, parallel array query optimization and execution, array language design, and time travel.The primary broader impact of SciDB is on the community of scientists who benefit from the tool. By keeping scientists "in the loop" in the design of the system from the outset, the project delivers software that is broadly usable to the community. The proposal also funds participation in a series of workshops that seek to engage even more of the science community. SciDB is an open-source effort, with an initial prototype (http://www.scidb.org/) already downloaded by hundreds of users. Finally, the PIs have a strong track record of delivering robust data management software that is widely used and involving students in the process, including students from under-represented groups. Further information can be found on the project web page (http://database.cs.brown.edu/projects/scidb).

该合作项目汇集了布朗大学（IIS-1111423）、华盛顿大学（IIS-1110370）、马萨诸塞州理工学院（IIS-1111371）、波特兰州立大学（IIS-1110917）和威斯康星大学麦迪逊分校（IIS-1111423）五个研究小组的专业知识。科学数据管理传统上使用文件系统执行，最好使用根据低级数据格式结构化的文件。 更高级别的数据管理基础设施是特定于任务的，并且不能在不同的领域中重用，这导致科学家在管理他们的数据方面花费了数百万美元的重复实施工作。 该项目的目标是开发一个科学数据库（SciDB），这是一个为科学应用而设计和优化的系统。SciDB的目标是为科学做关系数据库为商业世界做的事情，即提供适用于许多科学领域的高性能，商业质量和可扩展的数据管理系统。与现有的数据库系统相比，SciDB基于多维数组数据模型，包括多个特定于科学和对科学至关重要的功能：起源，不确定性，版本，时间旅行，科学特定的操作，以及现场数据处理。没有任何现有的系统在一个高度可扩展的引擎中提供所有这些功能。因此，SciDB除了支持领域科学家进行数据驱动的知识发现外，还显着推进了数据管理的最新技术。SciDB的智力价值在于为嵌套数组存储、并行数组查询优化和执行、数组语言设计和时间旅行探索新颖的高性能解决方案。SciDB的主要广泛影响是受益于该工具的科学家社区。 通过让科学家从一开始就参与系统的设计，该项目提供了可广泛用于社区的软件。该提案还资助参加一系列旨在吸引更多科学界人士参与的研讨会。SciDB是一个开源项目，其初始原型（http：//www.scidb.org/）已经被数百名用户下载。最后，PI在提供强大的数据管理软件方面有着良好的记录，该软件被广泛使用，并让学生参与到这一过程中，包括来自代表性不足群体的学生。更多信息可在项目网页上查阅（http：//database.cs.brown.edu/projects/scidb）。