Putting Astronomy's Head in the Cloud

点击翻译按钮获取中文摘要

Astrophysics is addressing many fundamental questions about the nature of the universe through a series of ambitious wide-field optical and infrared imaging surveys (e.g. studying the properties of dark matter and the nature of dark energy). To accomplish these goals requires new methodologies for analyzing and understanding petascale data sets (with the data being collected at a rate 1000x greater than current surveys). This research focusses on exploring an emerging paradigm for data intensive applications, map-reduce(using Hadoop for the implementation of map-reduce), and how it scales to the analysis of astronomical images. The work is addressing the efficiency of map-reduce for determining spatial and temporal overlaps between terabyte scale imaging data sets when compared to standard database techniques. We are delivering new algorithms for indexing, accessing and analyzing astronomical images using map-reduce that can balance the load between the compute nodes on distributed systems. We are also delivering applications that will analyze the spatial distribution of star formation within galaxies (combining large multispectral data sets) and for identifying asteroids within a time series of data where the asteroid may be below the detection threshold of any one image. This work will have a broad range of applications to any data intensive field.