III: Large: Collaborative Research: SciDB - An Array Oriented Data Management System for Massive Scale Scientific Data

该合作项目汇集了布朗大学（IIS-1111423）、华盛顿大学（IIS-1110370）、麻省理工学院（IIS-1111371）、波特兰州立大学（IIS-1110917）和威斯康星大学麦迪逊分校（IIS-1111423）五个研究团队的专业知识。科学数据管理传统上是使用文件系统进行的，最多使用根据低级数据格式构建的文件。更高级别的数据管理基础设施是特定于任务的，不能在不同的领域中重用，导致科学家们花费数百万美元来管理他们的数据。该项目的目标是开发一个科学数据库（SciDB），一个为科学应用而设计和优化的系统。SciDB的目标是为科学提供关系数据库为商业世界提供的服务，即为许多科学领域提供适合的高性能、商业质量和可扩展的数据管理系统。与现有数据库系统相比，SciDB基于多维数组数据模型，并包括科学特有的和对科学至关重要的多个特征：来源、不确定性、版本、时间旅行、科学特定操作和原位数据处理。没有现有的系统在一个单一的、高度可扩展的引擎中提供所有这些功能。因此，除了支持领域科学家进行数据驱动的知识发现之外，SciDB还显著地推进了数据管理方面的最新技术。SciDB的智力优势在于为嵌套数组存储、并行数组查询优化和执行、数组语言设计和时间旅行探索新颖的高性能解决方案。SciDB主要的更广泛的影响是对受益于该工具的科学家群体。通过让科学家从一开始就参与系统设计的“循环”，该项目交付的软件可以广泛地为社区所用。该提案还资助了一系列旨在吸引更多科学界人士参与的研讨会。SciDB是一个开源项目，其初始原型（http://www.scidb.org/）已经被数百名用户下载。最后，pi在提供广泛使用的强大数据管理软件方面有着良好的记录，并让学生参与到这一过程中，包括来自代表性不足群体的学生。更多信息可以在项目网页（http://database.cs.brown.edu/projects/scidb）上找到。

This collaborative project brings together expertise of five research teams at Brown University (IIS-1111423), University of Washington (IIS-1110370), Massachusetts Institute of Technology (IIS-1111371), Portland State University (IIS-1110917) and University of Wisconsin-Madison (IIS-1111423). Scientific data management has traditionally been performed using the file system, at best using files structured according to a low-level data format. Higher-level data management infrastructure has been task-specific and not reusable in different domains, resulting in millions of dollars of duplicated implementation effort by scientists to manage their data. The goal of this project is the development of a scientific database (SciDB), a system designed and optimized for scientific applications. The aim of SciDB is to do for science what relational databases did for the business world, namely to provide a high performance, commercial-quality and scalable data management system appropriate for many science domains.In contrast to existing database systems, SciDB is based on a multidimensional array data model and includes multiple features specific to science and critical for science: provenance, uncertainty, versions, time travel, science-specific operations, and in situ data processing. No existing system offers all these features in a single, highly scalable engine. SciDB thus significantly advances the state-of-the-art in data management in addition to supporting domain scientists in data-driven knowledge discovery. The intellectual merit of SciDB is in exploring novel, high performance solutions to nested array storage, parallel array query optimization and execution, array language design, and time travel.The primary broader impact of SciDB is on the community of scientists who benefit from the tool. By keeping scientists "in the loop" in the design of the system from the outset, the project delivers software that is broadly usable to the community. The proposal also funds participation in a series of workshops that seek to engage even more of the science community. SciDB is an open-source effort, with an initial prototype (http://www.scidb.org/) already downloaded by hundreds of users. Finally, the PIs have a strong track record of delivering robust data management software that is widely used and involving students in the process, including students from under-represented groups. Further information can be found on the project web page (http://database.cs.brown.edu/projects/scidb).