Efficient Astronomical Data Processing Among Distributed Astronomical Radio Observatories

分布式天文射电观测站的高效天文数据处理

• 批准号: 2206522
• 负责人: Haiying Shen
• 金额: $ 58.69万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2206522&HistoricalAwards=false
• 关键词: Efficient; Astronomical; Data; Processing; Among

Astronomy has become a data rich science and one of the biggest consumers of computing resources. NSF’s National Radio Astronomy Observatory (NRAO) collects and processes radio telescope data from a broad selection of facilities, but this centralized approach cannot provide timely analysis, driving an increasing need for an efficient astronomical data processing system among distributed observatories. As future telescope projects come online, the data size will grow dramatically and astronomical experiments will require more accurate data analysis and processing, and more detailed results. A critical challenge is job scheduling to maximize application performance and minimize system cost. This project will design and develop system software for geo-distributed observatories, focusing on astronomical data processing performance measurement and performance analysis, and efficient processing system design. Research results will be disseminated through open-source software releases. The project will be showcased in relevant courses and will provide thorough training and collaborative research opportunities to participating undergraduate, graduate, and K-12 students and faculty, and under-represented and female students.This software for scalable and efficient astronomical (SEA) data processing includes novel coordination between job scheduling, job directed acyclic graph (DAG) configuration, and the distribution of data around the globe. The team will analyze the performance of NRAO’s Common Astronomy Software Applications (CASA), and of machine learning (ML) and deep learning (DL) algorithms for astronomical research, on realistic datasets. Using those results, they will design ML/DL-assisted heuristic methods that adaptively determine the job DAG configuration and schedule, allowing an application to process geo-distributed datasets with maximum performance and minimum system cost.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

天文学已经成为一门数据丰富的科学，也是计算资源的最大消费者之一。美国国家科学基金会（NSF）的国家射电天文台（NRAO）从广泛选择的设施中收集和处理射电望远镜数据，但这种集中的方法不能提供及时的分析，这推动了对分布式天文台之间高效天文数据处理系统的日益增长的需求。随着未来的望远镜项目上线，数据量将急剧增长，天文实验将需要更准确的数据分析和处理，以及更详细的结果。一个关键的挑战是作业调度，以最大化应用程序性能和最小化系统成本。本项目将设计和开发地理分布式天文台系统软件，重点研究天文数据处理性能测量和性能分析，以及高效处理系统设计。研究成果将通过开源软件发布进行传播。该项目将在相关课程中展示，并将为参与的本科生，研究生，K-12学生和教师，以及代表性不足的学生和女性学生提供全面的培训和合作研究机会。该软件用于可扩展和高效的天文（SEA）数据处理，包括作业调度、作业有向无环图（DAG）配置和全球数据分布之间的新颖协调。该团队将分析NRAO的通用天文学软件应用程序（CASA）以及用于天文学研究的机器学习（ML）和深度学习（DL）算法在实际数据集上的性能。利用这些结果，他们将设计ML/ dl辅助的启发式方法，自适应地确定作业DAG配置和时间表，允许应用程序以最大的性能和最小的系统成本处理地理分布式数据集。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。