ST-CRTS: Enabling Processing of Large-Scale Scientific Data Through Compiler Supported XML Abstractions

背景分析大型和/或地理上分布的科学数据集，正在成为网格计算的一个关键组成部分，并为科学发现带来了巨大的希望。然而，这样的分析任务是复杂的低级和专门的数据存储格式，并缺乏标准的接口访问数据。科学数据集通常存储在二进制或字符平面文件中。这使得紧凑的存储和高效的处理成为可能，但是，使处理的规范变得更加困难。这个项目的总体动机是“未来的分布式和网格计算应用程序将需要高级抽象和编程语言来访问，检索，并处理以紧凑的低级格式物理存储的大规模科学数据集”.智力上的优点开发一个编译器来支持上述性质的系统涉及到许多挑战。这些挑战的出现主要是因为以下四个原因：1）支持高级抽象，这意味着编译器不仅需要生成正确访问数据的代码，而且还需要自动实现高局部性2）大型的驻留数据集，这使得生成具有高数据访问局部性的低级代码具有相当大的挑战性3）我们针对科学数据分析，数据挖掘，和图像分析应用程序，这些应用程序在重构编译器社区中没有得到太多的关注，4）XQuery的特性，因为XQuery是从函数式，声明式，以及数据库编程语言，由于它支持的结构和编程风格，出现了许多问题。更广泛的影响-PI正在致力于创建一个新的应用程序驱动的网格计算的两个季度序列。在这里，将强调的主题之一是分析从真实的生物和医学应用中产生的大型数据集。PI目前正在与三名女博士生合作，并希望其中一人参与该项目。Agrawal和他在俄亥俄州州立大学计算机科学与工程系的许多同事一起，参与了与俄亥俄州两个HBCU的合作计划。该合作计划包括：1）OSU教师的客座讲座和2）通过独立学习和暑期项目与OSU教师联合为HBCU本科生提供研究机会（我们希望能够使用NSF REU补充资金资助这项活动）。

BackgroundAnalysis of large and/or geographically distributed scientific datasets, is emerging as a key component of grid computing and holds great promise for scientific discoveries. However, such analysis tasks are complicated by low-level and specialized data storage formats, and lack of standard interfaces for accessing the data. Scientific datasets are typically stored in binary or character flat-files. These enable compact storage and efficient processing, but, make the specification of processing much harder.The overall motivation for this project is "future distributed and grid computing applications will need high-level abstractions and programming languages to access, retrieve, and process large-scale scientific datasets that are physically stored in compact low-level formats".Intellectual MeritDevelop a compiler to support a system of the nature described above involves a number of challenges.These challenges arise primarily because of the following four reasons:1) Supporting high-level abstractions, which implies that compiler not only needs to generate code that will correctly access data, but also needs to automatically achieve high-locality 2) Large Disk-Resident Datasets, which makes it quite challenging to generate low-level code that will havehigh data access locality3) Characteristics of the Applications we are targeting scientific data analysis, data mining, andimage analysis applications that have not received much attention in the restructuring compilers community, 4)Features of XQuery since XQuery is derived from functional, declarative, as well as database programming languages, a number of issues arise because of the constructs and programming styles it supports.Broader Impact-The PI is working on creating a new two quarter sequence on application-driven grid computing. Here, one of the topics that will be emphasized is the analysis of large datasets arising from real biological and medical applications.- The PI is currently working with three female Ph.D students, and expects to involve one of them in this project. Together with many of his colleagues in the Computer Science and Engineering department at the Ohio State University, Agrawal is part of a collaboration plan with two HBCUs in the state of Ohio. This collaboration plan includes: 1) Guest lectures by OSU facultyand 2) Research opportunities for HBCU undergraduates through independent study and summer projects jointly with OSU faculty (we hope to be able to use NSF REU supplements for funding this activity).