Efficient Astronomical Data Processing Among Distributed Astronomical Radio Observatories

点击翻译按钮获取中文摘要

Astronomy has become a data rich science and one of the biggest consumers of computing resources. NSF’s National Radio Astronomy Observatory (NRAO) collects and processes radio telescope data from a broad selection of facilities, but this centralized approach cannot provide timely analysis, driving an increasing need for an efficient astronomical data processing system among distributed observatories. As future telescope projects come online, the data size will grow dramatically and astronomical experiments will require more accurate data analysis and processing, and more detailed results. A critical challenge is job scheduling to maximize application performance and minimize system cost. This project will design and develop system software for geo-distributed observatories, focusing on astronomical data processing performance measurement and performance analysis, and efficient processing system design. Research results will be disseminated through open-source software releases. The project will be showcased in relevant courses and will provide thorough training and collaborative research opportunities to participating undergraduate, graduate, and K-12 students and faculty, and under-represented and female students.This software for scalable and efficient astronomical (SEA) data processing includes novel coordination between job scheduling, job directed acyclic graph (DAG) configuration, and the distribution of data around the globe. The team will analyze the performance of NRAO’s Common Astronomy Software Applications (CASA), and of machine learning (ML) and deep learning (DL) algorithms for astronomical research, on realistic datasets. Using those results, they will design ML/DL-assisted heuristic methods that adaptively determine the job DAG configuration and schedule, allowing an application to process geo-distributed datasets with maximum performance and minimum system cost.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.