Compiler and Runtime Support for Data Intensive Computing on Multi-dimensional Data

科学计算中最大、增长最快的问题之一是对超大数据集的分析和处理。这些科学数据集可以来自长期运行的模拟(例如，对水污染的模拟，在以后创建预期水情的“快照”)、遥感数据的档案(例如，高分辨率卫星图像)和医学图像的档案(例如，对一个病人或一组病人的核磁共振扫描)。这些数据集通常是多维的，包括空间坐标、时间戳和每个点的几个物理属性。现在有几个系统支持此类数据集的存储、检索和可视化，但很少有系统能够有效地处理这些数据。该项目将开发使用高级并行语言进行多维数据处理和分析的高效程序的方法。该项目将通过开发运行时例程来优化资源使用、适当的语言扩展以及针对大数据处理进行积极的编译器优化来解决这一问题。运行时方法将实施在大范围的大型数据集分析上优化计算效率的策略，同时考虑到数据的空间结构和分区以及要执行的计算。将这些例程合并到调查员的活动数据储存库中将大大推广和改进该系统。然后，语言扩展和编译器优化将利用运行时系统来使分析要在抽象级别表达的多维数据集的应用程序能够实现计算、存储和通信资源的高利用率。

One of the largest and fastest-growing problems in scientific computing is the analysis and processing of very large data sets. These scientific data sets can come from long-running simulations (e.g. simulations of water pollution that create "snapshots" of the expected water conditions at later times), archives of remote sensing data (e.g. high-resolution satellite imagery), and archives of medical images (e.g. MRI scans for a patient or group of patients). These data sets are usually multi-dimensional, including spatial coordinates, time stamps, and several physical properties at each point. Several systems now support storage, retrieval, and visualization of such data sets, but few can efficiently process the data. This project will develop methods to produce efficient programs to carry out multi-dimensional data processing and analysis using a high-level parallel language.The project will attack this problem by developing runtime routines for optimizing resource usage, appropriate language extensions, and aggressive compiler optimizations for large data processing. The runtime methods will implement policies that optimize computational efficiency on a broad range of large data set analyses, taking into account the spatial structure and partitioning of the data and the computation to be performed. Incorporating these routines into the investigator's Active Data Repository will substantially generalize and improve that system. The language extensions and compiler optimizations will then make use of the runtime system to enable applications that analyze multi-dimensional data sets to be expressed at an abstract level, yet achieve high utilization of computational, storage, and communication resources.