SHF:Small:Language Support for Ad Hoc Data Processing

SHF:Small:对即席数据处理的语言支持

• 批准号: 1016937
• 负责人: David Walker
• 金额: $ 50万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-08-15 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1016937&HistoricalAwards=false
• 关键词: SHF; Small; Language; Support; Ad

In every business, engineering endeavour and scientific discipline, workers are digitizing their knowledge with the hope of using computational methods to categorize, query, filter, search, diagnose, and visualize their data. While this effort is leading to remarkable industrial and scientific advances, it is also generating enormous amounts of ad hoc data (i.e., that data for which standard data processing tools such as query engines, statistical packages, graphing tools, or other software is not readily available). Ad hoc data poses tremendous challenges to its users because it is often highly varied, poorly documented, filled with errors, and continuously evolving --- yet ad hoc data also contains much valuable information. The goal of this research is to develop general-purpose software tools and techniques capable of managing ad hoc data efficiently. This research has the potential for a broad impact on society by dramatically improving the productivity of industrial data analysts, computer systems administrators and academics who must deal with ad hoc data on a day-to-day basis.The central technical challenge of the research involves designing, implementing and evaluating a new domain-specific programming language that facilitates the management of ad hoc data sets. This new programming language will allow data analysts to specify the structure of ad hoc data files, how those files are arranged in a file system and what meta-data is associated with them. Once a specification is complete, it will be possible to use it as documentation for the data set or for generating data-processing tools. The research will also involve developing new methods for enabling users to generate specifications quickly and accurately, without actually having to write down all of the details by hand. Finally, the research will develop new algorithms for implementing the generated data-processing tools efficiently.

在每一个企业、工程和科学领域，工作人员都在数字化他们的知识，希望使用计算方法来分类、查询、过滤、搜索、诊断和可视化他们的数据。虽然这一努力正在导致显着的工业和科学进步，它也产生了大量的特设数据（即，诸如查询引擎、统计包、绘图工具或其他软件的标准数据处理工具不容易获得的数据）。特别数据给它的用户带来了巨大的挑战，因为它经常是高度多样的，缺乏文档记录，充满错误，并不断发展-然而特别数据也包含许多有价值的信息。 本研究的目标是开发通用的软件工具和技术，能够有效地管理ad hoc数据。 这项研究有可能对社会产生广泛的影响，通过显着提高生产力的工业数据分析师，计算机系统管理员和学者谁必须处理ad hoc数据在日常的基础上，该研究的核心技术挑战涉及设计，实施和评估一个新的特定领域的编程语言，促进ad hoc数据集的管理。 这种新的编程语言将允许数据分析师指定特定数据文件的结构，这些文件在文件系统中的排列方式以及与它们相关的元数据。 规格一旦完成，就可以将其用作数据集或生成数据处理工具的文件。 这项研究还将涉及开发新的方法，使用户能够快速准确地生成规格，而不必实际上写下所有的细节。 最后，研究将开发新的算法，有效地实现生成的数据处理工具。