III: Large: Collaborative Research: SciDB - An Array Oriented Data Management System for Massive Scale Scientific Data

这个合作项目汇集了布朗大学(IIS-1111423)、华盛顿大学(IIS-1110370)、麻省理工学院(IIS-1111371)、波特兰州立大学(IIS-1110917)和威斯康星大学麦迪逊分校(IIS-1111423)的五个研究团队的专业知识。科学数据管理传统上是使用文件系统执行的，充其量使用根据低级数据格式构造的文件。更高级别的数据管理基础设施一直是特定于任务的，不能在不同的域中重复使用，导致科学家为管理他们的数据而重复实施数百万美元的工作。该项目的目标是开发一个科学数据库(SciDB)，这是一个为科学应用而设计和优化的系统。与现有的数据库系统不同，本数据库基于多维数组数据模型，具有多种科学特性：来源、不确定性、版本、时间旅行、科学特有的操作和现场数据处理。现有的任何系统都不能在一个高度可扩展的引擎中提供所有这些功能。因此，除了支持领域科学家进行数据驱动的知识发现外，本数据库还极大地推动了数据管理方面的最先进技术。SciDB的智能优势在于探索新的、高性能的解决方案来解决嵌套数组存储、并行数组查询优化和执行、数组语言设计和时间旅行。通过让科学家从一开始就了解系统设计的情况，该项目提供了可广泛用于社区的软件。该提案还为参加一系列研讨会提供了资金，这些研讨会试图让更多的科学界人士参与进来。本数据库是一个开源项目，最初的原型(http://www.scidb.org/))已经被数以百计的用户下载。最后，私人投资机构在提供被广泛使用并让学生参与这一过程的强大数据管理软件方面有着良好的记录，包括来自代表性不足群体的学生。欲了解更多信息，请访问项目网页(http://database.cs.brown.edu/projects/scidb).

This collaborative project brings together expertise of five research teams at Brown University (IIS-1111423), University of Washington (IIS-1110370), Massachusetts Institute of Technology (IIS-1111371), Portland State University (IIS-1110917) and University of Wisconsin-Madison (IIS-1111423). Scientific data management has traditionally been performed using the file system, at best using files structured according to a low-level data format. Higher-level data management infrastructure has been task-specific and not reusable in different domains, resulting in millions of dollars of duplicated implementation effort by scientists to manage their data. The goal of this project is the development of a scientific database (SciDB), a system designed and optimized for scientific applications. The aim of SciDB is to do for science what relational databases did for the business world, namely to provide a high performance, commercial-quality and scalable data management system appropriate for many science domains.In contrast to existing database systems, SciDB is based on a multidimensional array data model and includes multiple features specific to science and critical for science: provenance, uncertainty, versions, time travel, science-specific operations, and in situ data processing. No existing system offers all these features in a single, highly scalable engine. SciDB thus significantly advances the state-of-the-art in data management in addition to supporting domain scientists in data-driven knowledge discovery. The intellectual merit of SciDB is in exploring novel, high performance solutions to nested array storage, parallel array query optimization and execution, array language design, and time travel.The primary broader impact of SciDB is on the community of scientists who benefit from the tool. By keeping scientists "in the loop" in the design of the system from the outset, the project delivers software that is broadly usable to the community. The proposal also funds participation in a series of workshops that seek to engage even more of the science community. SciDB is an open-source effort, with an initial prototype (http://www.scidb.org/) already downloaded by hundreds of users. Finally, the PIs have a strong track record of delivering robust data management software that is widely used and involving students in the process, including students from under-represented groups. Further information can be found on the project web page (http://database.cs.brown.edu/projects/scidb).