Collaborative Research: The Dark Energy Survey Data Management System

AST-0813534/0812790/0813575Mohr/Jarvis/PeoplesThe暗能量探测(DES)是一项在2011年至2016年进行的中期探测，它将利用四项独立和互补的技术来研究暗能量的性质：(1)与美国国家科学基金会的南极望远镜Sunyaev-Zel‘dovich效应测绘实验合作进行的星系团探测；(2)宇宙切变测量；(3)红移壳层内的星系角星团测量(z=1.35)；以及(4)对大约2000颗1a型超新星(SNE Ia)的距离测量。DES将产生约170TB的原始数据，处理成可供科学使用的图像和目录，并共同添加到更深、更高质量的图像和目录中。总体而言，DES数据集将接近2PB。这一数据量和调查持续时间需要一种具有高度自动化和稳健性的新型数据管理系统，包括综合质量保证测试。目前的合作项目将建立DES数据管理(DESDM)系统。设计和测试了DESDM，使用了Mosaic2相机的模拟数据和实际数据。项目组已经创建了DESDM许多要素的工作原型，现在将把这些原型转化为DESDM系统的生产版本。DES是一个复杂的国际、多机构项目，对国内外影响深远。它被组织成DESDM和另外两个主题。这些是将安装在Cerro Tololo美洲天文台(CTIO)4米Blanco望远镜上的暗能量相机(DECAM)，以及CTIO的设施改进和集成项目(CTIOFP)。CTIOFP得到了国家科学基金会国家光学天文台的一个组成部分CTIO的支持，DECAM得到了能源部(DoE)以及国际和机构合作伙伴的支持。DESDM得到了NSF和项目合作伙伴的支持。DESDM项目正在与其他大型成像项目合作开发共同的解决方案，并将向社区提供所有生产的软件。学生将发挥重要作用，培养科学数据管理、高性能计算和科学分析工具开发方面的专业知识。DESDM系统将对DES数据进行存档，通过提供的科学分析界面实现公众访问和科学分析(在一年的专有期之后)，该界面将使用由伊利诺伊大学运营的NSF国家超级计算应用中心(NCSA)的高性能计算资源。本奖项未涵盖的项目的一个单独部分是DECAM减少门户，支持DECAM的非DES用户减少他们自己的数据。

AST-0813534/0812790/0813575Mohr/Jarvis/PeoplesThe Dark Energy Survey (DES) is an intermediate term survey, in the years 2011-2016, that will address the nature of dark energy using four independent and complementary techniques: (1) a galaxy cluster survey in collaboration with NSF's South Pole Telescope Sunyaev-Zel'dovich Effect mapping experiment, (2) a cosmic shear measurement, (3) a galaxy angular clustering measurement within redshift shells to z=1.35, and (4) distance measurements to approximately 2000 type 1a supernovae (SNe Ia). The DES will produce about 170Tb of raw data, processed into science-ready images and catalogs and co-added into deeper, higher quality images and catalogs. In total, the DES dataset will approach 2Pb. This data volume and the survey duration require a new type of data management system with a high degree of automation and robustness, including integrated quality assurance testing. The present collaborative project will build the DES Data Management (DESDM) system. DESDM has been designed and tested, using both simulated data and real data from the Mosaic2 camera. The project team has created working prototypes of many elements of DESDM and will now transform those prototypes into the production version of the DESDM system.DES is a complex international, multi-agency project, with far-reaching foreign and domestic impacts. It is organized into DESDM and two other themes. These are the Dark Energy Camera (DECam), to be installed at the 4-meter Blanco telescope of the Cerro Tololo Interamerican Observatory (CTIO), and the facilities improvement and integration project at the CTIO (CTIOFP). The CTIOFP is supported by CTIO, a component of NSF's National Optical Astronomy Observatory, and DECam is supported by the Department of Energy (DoE) and by international and institutional partners. DESDM is supported by NSF and also by the project partners.The DESDM project is working collaboratively with other large scale imaging projects to develop common solutions, and will be making all produced software available to the community. Students will play important role, developing expertise in aspects of scientific data management, high performance computing, and the development of science analysis tools. The DESDM system will archive the DES data, enabling public access and science analyses (after a one year proprietary period) through a provided scientific analysis interface that will use high performance computing resources at the NSF's National Center for Supercomputing Applications (NCSA), operated by the University of Illinois. A separate part of the project not covered by this award is a DECam Reduction Portal, supporting non-DES users of the DECam to reduce their own data.