ST-CRTS: Enabling Processing of Large-Scale Scientific Data Through Compiler Supported XML Abstractions

背景分析大型和/或地理分布的科学数据集，正在成为网格计算的关键组件，并为科学发现带来巨大的希望。然而，这种分析任务由于低级和专门的数据存储格式以及缺乏访问数据的标准接口而变得复杂。科学数据集通常存储在二进制或字符平面文件中。这使得紧凑的存储和高效的处理成为可能，但也使得处理的规范更加严格。该项目的总体动机是“未来的分布式和网格计算应用程序将需要高级抽象和编程语言来访问、检索和处理以紧凑的低级格式物理存储的大规模科学数据集”。开发编译器以支持上述性质的系统涉及许多挑战。这些挑战主要是由于以下四个原因而出现的：1)支持高级抽象，这意味着编译器不仅需要生成将正确访问数据的代码，但也需要自动实现高度局部性2)大型磁盘驻留数据集，这使得生成将具有高度数据访问局部性的低级代码变得非常具有挑战性3)我们所针对的应用程序的特征科学数据分析、数据挖掘和图像分析应用程序在重构编译器社区中没有得到太多关注，4)XQuery的功能由于XQuery派生自函数式、声明性以及数据库编程语言，因此产生了许多问题，因为它支持的构造和编程风格。在这里，将强调的主题之一是对来自真实生物和医学应用的大型数据集的分析。-PI目前正在与三名女性博士生合作，并希望让其中一人参与这个项目。阿格拉瓦尔和他在俄亥俄州立大学计算机科学与工程系的许多同事一起，参与了与俄亥俄州两所HBCU的合作计划。这一合作计划包括：1)俄亥俄州立大学教职员工的客座讲座；2)通过与俄勒冈州州立大学教职员工合作的独立学习和暑期项目，为HBCU本科生提供研究机会(我们希望能够使用NSF REU补充资金来资助这项活动)。

BackgroundAnalysis of large and/or geographically distributed scientific datasets, is emerging as a key component of grid computing and holds great promise for scientific discoveries. However, such analysis tasks are complicated by low-level and specialized data storage formats, and lack of standard interfaces for accessing the data. Scientific datasets are typically stored in binary or character flat-files. These enable compact storage and efficient processing, but, make the specification of processing much harder.The overall motivation for this project is "future distributed and grid computing applications will need high-level abstractions and programming languages to access, retrieve, and process large-scale scientific datasets that are physically stored in compact low-level formats".Intellectual MeritDevelop a compiler to support a system of the nature described above involves a number of challenges.These challenges arise primarily because of the following four reasons:1) Supporting high-level abstractions, which implies that compiler not only needs to generate code that will correctly access data, but also needs to automatically achieve high-locality 2) Large Disk-Resident Datasets, which makes it quite challenging to generate low-level code that will havehigh data access locality3) Characteristics of the Applications we are targeting scientific data analysis, data mining, andimage analysis applications that have not received much attention in the restructuring compilers community, 4)Features of XQuery since XQuery is derived from functional, declarative, as well as database programming languages, a number of issues arise because of the constructs and programming styles it supports.Broader Impact-The PI is working on creating a new two quarter sequence on application-driven grid computing. Here, one of the topics that will be emphasized is the analysis of large datasets arising from real biological and medical applications.- The PI is currently working with three female Ph.D students, and expects to involve one of them in this project. Together with many of his colleagues in the Computer Science and Engineering department at the Ohio State University, Agrawal is part of a collaboration plan with two HBCUs in the state of Ohio. This collaboration plan includes: 1) Guest lectures by OSU facultyand 2) Research opportunities for HBCU undergraduates through independent study and summer projects jointly with OSU faculty (we hope to be able to use NSF REU supplements for funding this activity).