Global Graphs: A Middleware for Data Intensive Computing

点击翻译按钮获取中文摘要

It is often the case that the time and effort required to develop effective and efficient software on high-end computing systems is the bottleneck in many areas of science and engineering. This project is building a novel middleware framework called Global Graphs that targets this bottleneck. Global Graphs takes a data-structure centric view of shared data where graph-based dynamic data structures drive the development of the rest of the system.A key scientific outcome of this proposed framework is to allow the programmer to have multiple views of the shared data as well as multiple views of the control and tasking model. This flexibility can be leveraged along a discrete scale of data and process views depending on whether the goal is to develop a quick prototype for validating ideas on small scale problems, or the goal is efficient realization on large scale problems, or something in between these two extremes. An additional outcome will be the development of a performance feedback engine that will provide the programmer insights into parts of the program to focus on for performance tuning.The proposed work has important implications for a range of domains requiring the processing of large scale datasets, including data mining, scientific computing and XML data management. The broader outcomes of this work will be to train capable undergraduate and graduate students. The PIs are actively encouraging under-represented minorities to participate in this effort.