KDI: Accessing Large Distributed Archives in Astronomy and Particle Physics

点击翻译按钮获取中文摘要

9980044SzalayThis proposal requests funding to support research into various ways of organizing and accessing the exceedingly large data sets now or soon to be acquired in astronomy and particle physics. The typical data acquisition rate for such experiments exceeds several Terabytes per year, and in the very near future will reach the Petabyte per year rate. Such amounts of data pose non-trivial challenges, in particular since they must be accessed quickly and conveniently by a large and geographically dispersed collaboration. The astronomy data includes (1) the Sloan Digital Sky Survey (SDSS), a 40 TB digital map of the northern sky; (2) data from the GALEX satellite (approx. 20 TB); and (3) data from the space shuttle based AMS experiment. In particle physics, data from the CMS experiment at the Large Hadron Collider (LHC) at the CERN laboratory in Geneva Switzerland will significantly exceed 1 PB per year.This research will address the following issues: (1) Organization of the data for efficient access. This will require exploration of several advanced data structures so that data may be stored in a fashion which anticipates future queries. (2) Storage of the data in widely distributed locations. This requires a novel approach from the perspective of both networking and distributed computing systems. Proposers will develop a middleware of Query Agents to perform the queries at the verious data locations. (3) Efficient handling of entire major archives. This requires development of parallel processing tools that can exploit arrays of commodity processors and disks to allow analysis of several Terabytes per hour. 9980044SzalayThis proposal requests funding to support research into various ways of organizing and accessing the exceedingly large data sets now or soon to be acquired in astronomy and particle physics. The typical data acquisition rate for such experiments exceeds several Terabytes per year, and in the very near future will reach the Petabyte per year rate. Such amounts of data pose non-trivial challenges, in particular since they must be accessed quickly and conveniently by a large and geographically dispersed collaboration. The astronomy data includes (1) the Sloan Digital Sky Survey (SDSS), a 40 TB digital map of the northern sky; (2) data from the GALEX satellite (approx. 20 TB); and (3) data from the space shuttle based AMS experiment. In particle physics, data from the CMS experiment at the Large Hadron Collider (LHC) at the CERN laboratory in Geneva Switzerland will significantly exceed 1 PB per year.This research will address the following issues: (1) Organization of the data for efficient access. This will require exploration of several advanced data structures so that data may be stored in a fashion which anticipates future queries. (2) Storage of the data in widely distributed locations. This requires a novel approach from the perspective of both networking and distributed computing systems. Proposers will develop a middleware of Query Agents to perform the queries at the verious data locations. (3) Efficient handling of entire major archives. This requires development of parallel processing tools that can exploit arrays of commodity processors and disks to allow analysis of several Terabytes per hour. 9980044SzalayThis proposal requests funding to support research into various ways of organizing and accessing the exceedingly large data sets now or soon to be acquired in astronomy and particle physics. The typical data acquisition rate for such experiments exceeds several Terabytes per year, and in the very near future will reach the Petabyte per year rate. Such amounts of data pose non-trivial challenges, in particular since they must be accessed quickly and conveniently by a large and geographically dispersed collaboration. The astronomy data includes (1) the Sloan Digital Sky Survey (SDSS), a 40 TB digital map of the northern sky; (2) data from the GALEX satellite (approx. 20 TB); and (3) data from the space shuttle based AMS experiment. In particle physics, data from the CMS experiment at the Large Hadron Collider (LHC) at the CERN laboratory in Geneva Switzerland will significantly exceed 1 PB per year.This research will address the following issues: (1) Organization of the data for efficient access. This will require exploration of several advanced data structures so that data may be stored in a fashion which anticipates future queries. (2) Storage of the data in widely distributed locations. This requires a novel approach from the perspective of both networking and distributed computing systems. Proposers will develop a middleware of Query Agents to perform the queries at the verious data locations. (3) Efficient handling of entire major archives. This requires development of parallel processing tools that can exploit arrays of commodity processors and disks to allow analysis of several Terabytes per hour.