ST-CRTS: Enabling Processing of Large-Scale Scientific Data Through Compiler Supported XML Abstractions

对大型和/或地理分布的科学数据集的分析，正在成为网格计算的一个关键组成部分，对科学发现有着巨大的希望。然而，由于低级和专门的数据存储格式，以及缺乏访问数据的标准接口，这些分析任务变得复杂。科学数据集通常存储在二进制或字符平面文件中。这些实现了紧凑的存储和高效的处理，但是，使处理规范变得更加困难。该项目的总体动机是“未来的分布式和网格计算应用程序将需要高级抽象和编程语言来访问、检索和处理以紧凑的低级格式物理存储的大规模科学数据集”。开发一个编译器来支持上述性质的系统涉及许多挑战。这些挑战的出现主要是因为以下四个原因：1)支持高级抽象，这意味着编译器不仅需要生成能够正确访问数据的代码，而且还需要自动实现高局部性2)大型磁盘驻留数据集，这使得生成具有高数据访问局部性的低级代码非常具有挑战性3)我们针对的应用程序的特点科学数据分析，数据挖掘，4) XQuery的特性由于XQuery是从函数式、声明式和数据库编程语言派生出来的，因此由于它所支持的结构和编程风格，出现了许多问题。更广泛的影响——PI正致力于在应用驱动的网格计算上创建一个新的二季度序列。在这里，将强调的主题之一是对来自实际生物和医学应用的大型数据集的分析。- PI目前正在与三名女博士生合作，并希望让其中一人参与该项目。与他在俄亥俄州立大学计算机科学与工程系的许多同事一起，Agrawal是俄亥俄州两所hbcu合作计划的一部分。该合作计划包括：1)由OSU教师客座授课；2)HBCU本科生通过自主学习和与OSU教师共同开展暑期项目的研究机会（我们希望能够使用NSF REU补助款资助这项活动）。

BackgroundAnalysis of large and/or geographically distributed scientific datasets, is emerging as a key component of grid computing and holds great promise for scientific discoveries. However, such analysis tasks are complicated by low-level and specialized data storage formats, and lack of standard interfaces for accessing the data. Scientific datasets are typically stored in binary or character flat-files. These enable compact storage and efficient processing, but, make the specification of processing much harder.The overall motivation for this project is "future distributed and grid computing applications will need high-level abstractions and programming languages to access, retrieve, and process large-scale scientific datasets that are physically stored in compact low-level formats".Intellectual MeritDevelop a compiler to support a system of the nature described above involves a number of challenges.These challenges arise primarily because of the following four reasons:1) Supporting high-level abstractions, which implies that compiler not only needs to generate code that will correctly access data, but also needs to automatically achieve high-locality 2) Large Disk-Resident Datasets, which makes it quite challenging to generate low-level code that will havehigh data access locality3) Characteristics of the Applications we are targeting scientific data analysis, data mining, andimage analysis applications that have not received much attention in the restructuring compilers community, 4)Features of XQuery since XQuery is derived from functional, declarative, as well as database programming languages, a number of issues arise because of the constructs and programming styles it supports.Broader Impact-The PI is working on creating a new two quarter sequence on application-driven grid computing. Here, one of the topics that will be emphasized is the analysis of large datasets arising from real biological and medical applications.- The PI is currently working with three female Ph.D students, and expects to involve one of them in this project. Together with many of his colleagues in the Computer Science and Engineering department at the Ohio State University, Agrawal is part of a collaboration plan with two HBCUs in the state of Ohio. This collaboration plan includes: 1) Guest lectures by OSU facultyand 2) Research opportunities for HBCU undergraduates through independent study and summer projects jointly with OSU faculty (we hope to be able to use NSF REU supplements for funding this activity).